Re: New GUC autovacuum_max_threshold ?

2024-04-26 Thread Joe Conway
On 4/26/24 09:31, Robert Haas wrote: On Fri, Apr 26, 2024 at 9:22 AM Joe Conway wrote: Although I don't think 50 is necessarily too small. In my view, having autovac run very quickly, even if more frequently, provides an overall better user experience. Can you elaborate on why you think

Re: New GUC autovacuum_max_threshold ?

2024-04-26 Thread Joe Conway
, having autovac run very quickly, even if more frequently, provides an overall better user experience. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Table AM Interface Enhancements

2024-04-10 Thread Joe Conway
need. It is extremely frustrating. But the solution is not to commit anyway and then blame the other people for not providing feedback. +many -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Security lessons from liblzma

2024-04-09 Thread Joe Conway
certain certifications. Of course there is no guarantee that such reviews would catch everything, but maybe we could establish post commit reviews by contributors in a more rigorous way? Granted, getting more qualified volunteers is not a trivial problem... -- Joe Conway PostgreSQL Contributors

Re: PostgreSQL 17 Release Management Team & Feature Freeze

2024-04-08 Thread Joe Conway
that lesson the hard way. I'm just distressed at our utter failure to learn from experience. I don't dispute that we could do better, and this is just a simplistic look based on "number of commits per day", but the attached does put it in perspective to some extent. -- Joe Conway

Re: Security lessons from liblzma

2024-03-31 Thread Joe Conway
On 3/31/24 11:49, Tom Lane wrote: Joe Conway writes: I am saying maybe those patches should be eliminated in favor of our tree including build options that would produce the same result. I don't really see how that can be expected to work sanely. It turns the responsibility for platform

Re: Security lessons from liblzma

2024-03-31 Thread Joe Conway
On 3/30/24 21:52, Bruce Momjian wrote: On Sat, Mar 30, 2024 at 07:54:00PM -0400, Joe Conway wrote: Virtually every RPM source, including ours, contains out of tree patches that get applied on top of the release tarball. At least for the PGDG packages, it would be nice to integrate them into our

Re: Security lessons from liblzma

2024-03-30 Thread Joe Conway
On 3/30/24 19:54, Joe Conway wrote: On 2024-03-30 16:50:26 -0400, Robert Haas wrote: or what Tom does when he builds the release tarballs. Tom follows this, at least last time I checked: https://wiki.postgresql.org/wiki/Release_process Reading through that, I wonder if this part is true

Re: Security lessons from liblzma

2024-03-30 Thread Joe Conway
of assuming that bad things can't happen to us. +1 and again with the +1 ;-) -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Popcount optimization using AVX512

2024-03-25 Thread Joe Conway
ards would be gratefully accepted... -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Possibility to disable `ALTER SYSTEM`

2024-03-19 Thread Joe Conway
scribes the intended use case.) I agree with pretty much all of this. +1 me too. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Improving contrib/tablefunc's error reporting

2024-03-09 Thread Joe Conway
On 3/9/24 15:39, Tom Lane wrote: Joe Conway writes: On 3/9/24 13:07, Tom Lane wrote: BTW, while I didn't touch it here, it seems fairly bogus that connectby() checks both type OID and typmod for its output columns while crosstab() only checks type OID. I think crosstab() is in the wrong

Re: Improving contrib/tablefunc's error reporting

2024-03-09 Thread Joe Conway
On 3/9/24 13:07, Tom Lane wrote: Joe Conway writes: On 3/5/24 17:04, Tom Lane wrote: After reading the thread at [1], I could not escape the feeling that contrib/tablefunc's error reporting is very confusing. The changes all look good to me and indeed more consistent with the docs. Do you

Re: Improving contrib/tablefunc's error reporting

2024-03-09 Thread Joe Conway
() checks both type OID and typmod for its output columns while crosstab() only checks type OID. I think crosstab() is in the wrong and needs to be checking typmod. That might be fit material for a separate patch though. I can take a look at this. Presumably this would not be for backpatching. -- Joe

Re: Emitting JSON to file using COPY TO

2024-03-08 Thread Joe Conway
On 3/8/24 12:28, Andrey M. Borodin wrote: Hello everyone! Thanks for working on this, really nice feature! On 9 Jan 2024, at 01:40, Joe Conway wrote: Thanks -- will have a look Joe, recently folks proposed a lot of patches in this thread that seem like diverted from original way

Re: Improving contrib/tablefunc's error reporting

2024-03-05 Thread Joe Conway
typmod. That might be fit material for a separate patch though. Been a long time since I gave contrib/tablefunc any love I guess ;-) I will have a look at your patches, and the other issue you mention, but it might be a day or three before I can give it some quality time. -- Joe Conway

PostgreSQL Contributors Updates

2024-03-03 Thread Joe Conway
of the PostgreSQL Contributors Team -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: An improved README experience for PostgreSQL

2024-02-28 Thread Joe Conway
the conversion even simpler. That's a pretty convincing proof-of-concept. Let's just do this, and then make sure to keep the file legible as plain text. +1 -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: An improved README experience for PostgreSQL

2024-02-28 Thread Joe Conway
. +1 Markdown is pretty readable as text, I'm not sure why we need both. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Joe Conway
On 2/19/24 13:13, Andres Freund wrote: On 2024-02-19 09:19:16 -0500, Joe Conway wrote: Couldn't we scale the rounding, e.g. allow small allocations as we do now, but above some number always round? E.g. maybe >= 2GB round to the nearest 256MB, >= 4GB round to the nearest 512MB, >=

Re: PGC_SIGHUP shared_buffers?

2024-02-19 Thread Joe Conway
do now, but above some number always round? E.g. maybe >= 2GB round to the nearest 256MB, >= 4GB round to the nearest 512MB, >= 8GB round to the nearest 1GB, etc? -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Replace current implementations in crypt() and gen_salt() to OpenSSL

2024-02-16 Thread Joe Conway
, + (errmsg("not available when using OpenSSL in FIPS mode"))); Makes sense +1 -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: pgjdbc is not working with PKCS8 certificates with password

2024-02-07 Thread Joe Conway
reset at org.postgresql.ssl.MakeSSL.convert(MakeSSL.java:43) at org.postgresql.core.v3.ConnectionFactoryImpl.enableSSL(ConnectionFactoryImpl.java:584) at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:168) / /.../ Regards, Madhu -- J

Re: [17] CREATE SUBSCRIPTION ... SERVER

2024-01-15 Thread Joe Conway
sense to use quote_literal_cstr() rather than defining your own appendEscapedValue() function? -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: System username in pg_stat_activity

2024-01-10 Thread Joe Conway
field name? (because if we keep auth_method:identity it's not really the authname anyway). I was worried system_user or sysuser would both be confusing with the fact that we have usesysid -- which would reference a *different* sys... I think if it is exactly "system_user" it wo

Re: Emitting JSON to file using COPY TO

2024-01-08 Thread Joe Conway
On 1/8/24 14:36, Dean Rasheed wrote: On Thu, 7 Dec 2023 at 01:10, Joe Conway wrote: The attached should fix the CopyOut response to say one column. Playing around with this, I found a couple of cases that generate an error: COPY (SELECT 1 UNION ALL SELECT 2) TO stdout WITH (format json

Re: Password leakage avoidance

2024-01-07 Thread Joe Conway
On 1/6/24 15:10, Tom Lane wrote: Joe Conway writes: The only code specific comments were Tom's above, which have been addressed. If there are no serious objections I plan to commit this relatively soon. I had not actually read this patchset before, but now I have, and I have a few minor

Re: Password leakage avoidance

2024-01-06 Thread Joe Conway
On 1/6/24 13:16, Sehrope Sarkuni wrote: On Sat, Jan 6, 2024 at 12:39 PM Joe Conway <mailto:m...@joeconway.com>> wrote: The only code specific comments were Tom's above, which have been addressed. If there are no serious objections I plan to commit this relatively soon.

Re: Password leakage avoidance

2024-01-06 Thread Joe Conway
On 12/24/23 10:14, Joe Conway wrote: On 12/23/23 11:00, Tom Lane wrote: Joe Conway writes: The attached patch set moves the guts of \password from psql into the libpq client side -- PQchangePassword() (patch 0001). Haven't really read the patch, just looked at the docs, but here's a bit

Re: Password leakage avoidance

2024-01-06 Thread Joe Conway
to this discussion, because the new function is called PQchangePassword(). The function PQencryptPasswordConn() has been around for a while and the horse is out of the gate on that one. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: BUG #17946: LC_MONETARY & DO LANGUAGE plperl - BUG

2024-01-05 Thread Joe Conway
marking the CF entry RwF for now. It can always be reopened, if Joe or Tristan or Heikki or someone else picks it up again. It is definitely a bug, so I do plan to get back to it at some point, hopefully sooner rather than later... -- Joe Conway PostgreSQL Contributors Team RDS Open Source

Re: SET ROLE x NO RESET

2023-12-31 Thread Joe Conway
that too, but see it as a separate feature. FWIW that is also supported by the set_user extension referenced elsewhere on this thread. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Build versionless .so for Android

2023-12-31 Thread Joe Conway
how long that is) Such discussions are adapted in a commit fest entry. No idea if there is a cooldown period after account creation before being able to touch the CF app contents, though. FWIW I have expedited the cooldown period, so Matthias can log in right away. -- Joe Conway PostgreSQL

Re: SET ROLE x NO RESET

2023-12-30 Thread Joe Conway
sion exits. ----- -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Password leakage avoidance

2023-12-27 Thread Joe Conway
On 12/27/23 16:09, Tom Lane wrote: Joe Conway writes: On 12/27/23 15:39, Peter Eisentraut wrote: On 23.12.23 16:13, Joe Conway wrote: The attached patch set moves the guts of \password from psql into the libpq client side -- PQchangePassword() (patch 0001). I don't follow how you get from

Re: Password leakage avoidance

2023-12-27 Thread Joe Conway
On 12/27/23 15:39, Peter Eisentraut wrote: On 23.12.23 16:13, Joe Conway wrote: I have recently, once again for the umpteenth time, been involved in discussions around (paraphrasing) "why does Postgres leak the passwords into the logs when they are changed". I know well that the

Re: Password leakage avoidance

2023-12-24 Thread Joe Conway
On 12/24/23 12:22, Tom Lane wrote: Joe Conway writes: Completely unrelated process bikeshedding: I changed the naming scheme I used for the split patch-set this time. I don't know if we have a settled/documented pattern for such naming, but the original pattern which I borrowed from someone

Re: Password leakage avoidance

2023-12-24 Thread Joe Conway
On 12/23/23 11:00, Tom Lane wrote: Joe Conway writes: The attached patch set moves the guts of \password from psql into the libpq client side -- PQchangePassword() (patch 0001). Haven't really read the patch, just looked at the docs, but here's a bit of bikeshedding: Thanks! * This seems

Password leakage avoidance

2023-12-23 Thread Joe Conway
ion when setting a new password. I will register this in the upcoming commitfest, but meantime thought/comments/etc. would be gratefully received. Thanks, -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.comdiff --git a/src/i

Re: Eager page freeze criteria clarification

2023-12-21 Thread Joe Conway
On 12/21/23 10:56, Melanie Plageman wrote: On Sat, Dec 9, 2023 at 9:24 AM Joe Conway wrote: However, even if we assume a more-or-less normal distribution, we should consider using subgroups in a way similar to Statistical Process Control[1]. The reasoning is explained in this quote

Re: Eager page freeze criteria clarification

2023-12-09 Thread Joe Conway
he shape of the population distribution. Therefore, as your subgroup size increases, your control chart limits will narrow, making the chart more sensitive to special cause variation and more prone to false alarms. [1] https://www.qualitygurus.com/rational-subgrouping-in-statistical-process-con

Re: Emitting JSON to file using COPY TO

2023-12-08 Thread Joe Conway
On 12/8/23 14:45, Daniel Verite wrote: Joe Conway wrote: copyto_json.007.diff When the source has json fields with non-significant line feeds, the COPY output has these line feeds too, which makes the output incompatible with rule #2 at https://jsonlines.org ("2. Each

Re: Emitting JSON to file using COPY TO

2023-12-07 Thread Joe Conway
ot;:1,"i":1} ,{"?column?":1,"i":2} ,{"?column?":1,"i":3} ] 8<-- -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Emitting JSON to file using COPY TO

2023-12-07 Thread Joe Conway
On 12/7/23 08:52, Joe Conway wrote: Or maybe this is preferred? 8<-- [{"ss":{"f1":1,"f2":1}}, {"ss":{"f1":1,"f2":2}}, {"ss":{"f1":1,"f2":3}}] 8<-- I don't know

Re: Emitting JSON to file using COPY TO

2023-12-07 Thread Joe Conway
On 12/7/23 08:35, Daniel Verite wrote: Joe Conway wrote: The attached should fix the CopyOut response to say one column. I.e. it ought to look something like: Spending more time with the doc I came to the opinion that in this bit of the protocol, in CopyOutResponse (B) ... Int16

Re: Emitting JSON to file using COPY TO

2023-12-07 Thread Joe Conway
test was over indexing on that one type. I am sure there are other micro-optimizations to be made here, but I also think that it is outside the scope of the COPY TO JSON patch. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 20:09, David G. Johnston wrote: On Wed, Dec 6, 2023 at 5:57 PM Joe Conway <mailto:m...@joeconway.com>> wrote: On 12/6/23 19:39, David G. Johnston wrote: > On Wed, Dec 6, 2023 at 4:45 PM Joe Conway mailto:m...@joeconway.com> > <mailto:m...@jo

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 18:09, Joe Conway wrote: On 12/6/23 14:47, Joe Conway wrote: On 12/6/23 13:59, Daniel Verite wrote: Andrew Dunstan wrote: IMNSHO, we should produce either a single JSON document (the ARRAY case) or a series of JSON documents, one per row (the LINES case). "

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 19:39, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:45 PM Joe Conway <mailto:m...@joeconway.com>> wrote: " The backend sends a CopyOutResponse message to the frontend, followed     by zero or more CopyData messages (always one per row), followed by

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 18:38, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:28 PM David G. Johnston mailto:david.g.johns...@gmail.com>> wrote: On Wed, Dec 6, 2023 at 4:09 PM Joe Conway mailto:m...@joeconway.com>> wrote: On 12/6/23 14:47, Joe Conway wrote: > O

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 18:28, David G. Johnston wrote: On Wed, Dec 6, 2023 at 4:09 PM Joe Conway <mailto:m...@joeconway.com>> wrote: On 12/6/23 14:47, Joe Conway wrote: > On 12/6/23 13:59, Daniel Verite wrote: >>      Andrew Dunstan wrote: >> >>>

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 14:47, Joe Conway wrote: On 12/6/23 13:59, Daniel Verite wrote: Andrew Dunstan wrote: IMNSHO, we should produce either a single JSON document (the ARRAY case) or a series of JSON documents, one per row (the LINES case). "COPY Operations" in the doc says: &qu

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 16:42, Sehrope Sarkuni wrote: On Wed, Dec 6, 2023 at 4:29 PM Joe Conway <mailto:m...@joeconway.com>> wrote: > 1. Outputting a top level JSON object without the additional column > keys. IIUC, the top level keys are always the column names. A common use

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
ow defaulting missing fields to NULL. good to record the ask, but applies to a different feature (COPY FROM instead of COPY TO). -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
performance left on the table to go after, no? -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
ta correctly despite the extra CopyData messages. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 10:44, Tom Lane wrote: Joe Conway writes: I believe this is ready to commit unless there are further comments or objections. I thought we were still mostly at proof-of-concept stage? The concept is narrowly scoped enough that I think we are homing in on the final patch

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 10:32, Andrew Dunstan wrote: On 2023-12-06 We 08:49, Joe Conway wrote: On 12/6/23 07:36, Andrew Dunstan wrote: On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12

Re: Emitting JSON to file using COPY TO

2023-12-06 Thread Joe Conway
On 12/6/23 07:36, Andrew Dunstan wrote: On 2023-12-05 Tu 16:46, Joe Conway wrote: On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/5/23 16:20, Andrew Dunstan wrote: On 2023-12-05 Tu 16:09, Joe Conway wrote: On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why you would have them. Oh I didn't address this -- I saw examples in the interwebs

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/5/23 16:12, Andrew Dunstan wrote: On 2023-12-05 Tu 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: On 2023-12-05 Tu 14:50, Davin Shearer wrote: Hi Joe, In reviewing the 005 patch, I think that when used with FORCE ARRAY, we should also _imply_ FORCE ROW DELIMITER

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/5/23 16:02, Joe Conway wrote: On 12/5/23 15:55, Andrew Dunstan wrote: and in any other case (e.g. LINES) I can't see why you would have them. Oh I didn't address this -- I saw examples in the interwebs of MSSQL server I think [1] which had the non-array with commas import and export

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
tting row delimiters false while force_array is true here: 8<--- + if (opts_out->force_array && + force_row_delimiter_specified && + !opts_out->force_row_delimiter) + ereport(ER

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
t;id":3,"f1":"aaa/bbb","f2":null} {"id":4,"f1":"aaa\","f2":null} {"id":5,"f1":"aaa\fbbb","f2":null} {"id":6,"f1":"aaa\nbbb","f2":null} {"id&q

Re: Emitting JSON to file using COPY TO

2023-12-05 Thread Joe Conway
On 12/4/23 21:54, Joe Conway wrote: On 12/4/23 17:55, Davin Shearer wrote: There are however a few characters that need to be escaped 1. |"|(double quote) 2. |\|(backslash) 3. |/|(forward slash) 4. |\b|(backspace) 5. |\f|(form feed) 6. |\n|(new line) 7. |\r|(carriage return) 8

Re: introduce dynamic shared memory registry

2023-12-05 Thread Joe Conway
rete plans to use this for anything, but I thought it might be useful for extensions for caching, etc. and wanted to see whether there was any interest in the feature. Notwithstanding any dragons there may be, and not having actually looked at the the patches, I love the concept! + -- Joe Conway

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Joe Conway
t;,"filler":1} {"style":"Unix","test":"abc\ndef","filler":2} {"style":"Mac","test":"abc\rdef","filler":3} {"style":"esc\\ape","test":"a\\r\\\r\\\n\\nb&quo

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Joe Conway
On 12/4/23 09:25, Andrew Dunstan wrote: On 2023-12-04 Mo 08:37, Joe Conway wrote: On 12/4/23 07:41, Andrew Dunstan wrote: On 2023-12-03 Su 20:14, Joe Conway wrote: (please don't top quote on the Postgres lists) On 12/3/23 17:38, Davin Shearer wrote: " being quoted as \\" break

Re: Emitting JSON to file using COPY TO

2023-12-04 Thread Joe Conway
On 12/4/23 07:41, Andrew Dunstan wrote: On 2023-12-03 Su 20:14, Joe Conway wrote: (please don't top quote on the Postgres lists) On 12/3/23 17:38, Davin Shearer wrote: " being quoted as \\" breaks the JSON. It needs to be \".  This has been my whole problem with COPY TO for

Re: Emitting JSON to file using COPY TO

2023-12-03 Thread Joe Conway
Davin -- how did you work around the issue with the way the built in functions output JSON? Andrew -- comments/thoughts? Joe On Sun, Dec 3, 2023, 10:51 Joe Conway <mailto:m...@joeconway.com>> wrote: On 12/3/23 10:31, Davin Shearer wrote: > Please be sure to include

Re: Emitting JSON to file using COPY TO

2023-12-03 Thread Joe Conway
On 12/3/23 14:52, Andrew Dunstan wrote: On 2023-12-03 Su 14:24, Joe Conway wrote: On 12/3/23 11:03, Joe Conway wrote: On 12/3/23 10:10, Andrew Dunstan wrote: I  realize this is just a POC, but I'd prefer to see composite_to_json() not exposed. You could use the already public datum_to_json

Re: Emitting JSON to file using COPY TO

2023-12-03 Thread Joe Conway
On 12/3/23 11:03, Joe Conway wrote: On 12/3/23 10:10, Andrew Dunstan wrote: I  realize this is just a POC, but I'd prefer to see composite_to_json() not exposed. You could use the already public datum_to_json() instead, passing JSONTYPE_COMPOSITE and F_RECORD_OUT as the second and third

Re: Emitting JSON to file using COPY TO

2023-12-03 Thread Joe Conway
On 12/3/23 11:03, Joe Conway wrote: From your earlier post, regarding constructing the aggregate -- not extensive testing but one data point: 8<-- test=# copy foo to '/tmp/buf' (format json, force_array); COPY 1000 Time: 36353.153 ms (00:36.353) test=# copy (sel

Re: Emitting JSON to file using COPY TO

2023-12-03 Thread Joe Conway
On 12/3/23 10:10, Andrew Dunstan wrote: On 2023-12-01 Fr 14:28, Joe Conway wrote: On 11/29/23 10:32, Davin Shearer wrote: Thanks for the responses everyone. I worked around the issue using the `psql -tc` method as Filip described. I think it would be great to support writing JSON using

Re: Emitting JSON to file using COPY TO

2023-12-03 Thread Joe Conway
"line with ' in it: 2456094","f2":"2023-12-03T10:44:40.971225-05:00"} {"id":2456095,"f1":"line with \\" in it: 2456095","f2":"2023-12-03T10:44:40.971228-05:00"} -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: Emitting JSON to file using COPY TO

2023-12-03 Thread Joe Conway
On 12/2/23 17:37, Joe Conway wrote: On 12/2/23 16:53, Nathan Bossart wrote: On Sat, Dec 02, 2023 at 10:11:20AM -0500, Tom Lane wrote: So if you are writing a production that might need to match FORMAT followed by JSON, you need to match FORMAT_LA too. Thanks for the pointer. That does seem

Re: Emitting JSON to file using COPY TO

2023-12-02 Thread Joe Conway
On 12/2/23 13:50, Maciek Sakrejda wrote: On Fri, Dec 1, 2023 at 11:32 AM Joe Conway wrote: 1. Is supporting JSON array format sufficient, or does it need to support some other options? How flexible does the support scheme need to be? "JSON Lines" is a semi-standard format

Re: Emitting JSON to file using COPY TO

2023-12-02 Thread Joe Conway
); } +| FORMAT_LA copy_generic_opt_arg +{ +$$ = makeDefElem("format", $2, @1); +} ; copy_generic_opt_arg: Yep -- I concluded the same. Thanks Tom! -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases

Re: Emitting JSON to file using COPY TO

2023-12-02 Thread Joe Conway
On 12/1/23 18:09, Nathan Bossart wrote: On Fri, Dec 01, 2023 at 02:28:55PM -0500, Joe Conway wrote: I did a quick PoC patch (attached) -- if there interest and no hard objections I would like to get it up to speed for the January commitfest. Cool. I would expect there to be interest, given

Re: Emitting JSON to file using COPY TO

2023-12-01 Thread Joe Conway
for processing, and NiFi has superior handling of JSON (via JOLT parsers) versus CSV where parsing is generally done with regex.  I want to be able to emit JSON using a postgres function and thus COPY TO. Definitely +1 for COPY TO. I don't think COPY FROM will work out well unless the JSON is requ

Re: Emitting JSON to file using COPY TO

2023-12-01 Thread Joe Conway
does it need to support some other options? How flexible does the support scheme need to be? 2. This only supports COPY TO and we would undoubtedly want to support COPY FROM for JSON as well, but is that required from the start? Thanks for any feedback. -- Joe Conway PostgreSQL Contributors T

Re: How to solve the problem of one backend process crashing and causing other processes to restart?

2023-11-13 Thread Joe Conway
his does nothing to prevent OOM kills, which are becoming more prevalent as, for example, running Postgres in a container (or otherwise) with a cgroup memory limit becomes more popular. And in any case, there are enterprise use cases that necessarily avoid Postgres due to this behavior, which is

Re: commitfest app down for repairs

2023-09-30 Thread Joe Conway
On 9/30/23 08:47, Joe Conway wrote: The committfest app is down for repairs. We will reply back here once it is back up. The commitfest app is back up. We restored to a backup from one day prior. We will take a look at what changed in between, but it might be up to folks to redo some things

commitfest app down for repairs

2023-09-30 Thread Joe Conway
The committfest app is down for repairs. We will reply back here once it is back up. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-09-25 Thread Joe Conway
On 9/25/23 14:03, Jeff Davis wrote: On Mon, 2023-09-25 at 12:00 -0400, Joe Conway wrote: Should there be a way to have a separate "execution" search_path? I hadn't considered that and I like that idea for a few reasons: * a lot of the problem cases are for functions that

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-09-25 Thread Joe Conway
qualifying of one's own functions), but agree that almost no one wants that for the latter (needing to schema qualify every table reference). Should there be a way to have a separate "execution" search_path? -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databa

Re: RFC: pg_stat_logmsg

2023-09-13 Thread Joe Conway
On 7/9/23 14:13, Joe Conway wrote: On 7/7/23 01:38, Gurjeet Singh wrote: In this case, the error message stored in pg_stat_logmsg is just '%s'. The filename and line number columns might help identify the actual error but it requires users to read the source code and cannot know the actual

Re: Possibility to disable `ALTER SYSTEM`

2023-09-07 Thread Joe Conway
://github.com/pgaudit/set_user -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: [17] CREATE SUBSCRIPTION ... SERVER

2023-08-31 Thread Joe Conway
precedent for that and it seems to violate the idea of an "extension" entirely. Can you explain more concretely how we might resolve that? Maybe move postgres_fdw to be a first class built in feature instead of an extension? -- Joe Conway PostgreSQL Contributors Team RDS Open Sou

Re: BUG #17946: LC_MONETARY & DO LANGUAGE plperl - BUG

2023-08-27 Thread Joe Conway
On 8/15/23 10:40, Heikki Linnakangas wrote: On 01/08/2023 16:48, Joe Conway wrote: Any further comments on the posted patch[1]? I would like to apply/push this prior to the beta and minor releases next week. I'm not sure about the placement of the uselocale() calls. In plperl_spi_exec

Re: list of acknowledgments for PG16

2023-08-22 Thread Joe Conway
L docs and put them on the website. These days the docs update is part of a scripted process for doing an entire release. I'm sure we could figure out how to just release the updated docs, but with RC1 a week away, is it really worthwhile? -- Joe Conway PostgreSQL Contributors Team RDS Open

Re: C function to return double precision[][]

2023-08-21 Thread Joe Conway
generic to the element type (i.e. not specifically float8), but the needed type conversion stuff happens in here: https://github.com/postgres-plr/plr/blob/20a1f133bcf2bc8f37ac23da191aea590d612619/plr.c#L1109 HTH, -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web

Re: Would it be possible to backpatch Close support in libpq (28b5726) to PG16?

2023-08-16 Thread Joe Conway
, would we add libpq support for Close to 16.2? Seems pretty clearly a "no" to me. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-08-12 Thread Joe Conway
On 8/12/23 09:15, Joe Conway wrote: On 8/11/23 22:35, Jeff Davis wrote: 2. We can more accurately serve the user's intent. For instance, the safe search_path of "pg_catalog, pg_temp" is arcane and seems to be there just because we don't have a way to specify that pg_temp be exclude

Re: CREATE FUNCTION ... SEARCH { DEFAULT | SYSTEM | SESSION }

2023-08-12 Thread Joe Conway
freedom to do that. Personally I think having pg_temp in the SYSTEM search path makes sense for temp tables, but I find it easy to forget that functions can be created by unprivileged users in pg_temp, and therefore having pg_temp in the search path for functions is dangerous. -- Joe Conway Post

Re: BUG #17946: LC_MONETARY & DO LANGUAGE plperl - BUG

2023-08-01 Thread Joe Conway
On 7/3/23 12:25, Tristan Partin wrote: On Sat Jun 24, 2023 at 8:09 AM CDT, Joe Conway wrote: Although I have not looked yet, presumably we could have similar problems with plpython. I would like to get agreement on this approach against plperl before diving into that though. Thoughts? I

Re: pgsql: Fix search_path to a safe value during maintenance operations.

2023-07-31 Thread Joe Conway
further. Even if the search_path does not change from the originally intended one, a newly created function can shadow the intended one based on argument coercion rules. -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: COPY table FROM STDIN via SPI

2023-07-12 Thread Joe Conway
On 7/12/23 14:43, c...@anastigmatix.net wrote: On 2023-07-12 14:18, Joe Conway wrote: On 7/11/23 22:52, James Sewell wrote: What about running a COPY directly from C - is that possible? https://www.postgresql.org/docs/current/libpq-copy.html Or is the question about a COPY kicked off from

Re: COPY table FROM STDIN via SPI

2023-07-12 Thread Joe Conway
-copy.html -- Joe Conway PostgreSQL Contributors Team RDS Open Source Databases Amazon Web Services: https://aws.amazon.com

Re: RFC: pg_stat_logmsg

2023-07-09 Thread Joe Conway
On 7/7/23 01:38, Gurjeet Singh wrote: On Thu, Jul 6, 2023 at 12:37 AM Masahiko Sawada wrote: On Sat, Jul 1, 2023 at 8:57 AM Joe Conway wrote: > > The basic idea is to mirror how pg_stat_statements works, except the > logged messages keyed by filename, lineno, and elevel

  1   2   3   4   >