Re: Having query cache in core

2018-05-07 Thread Heikki Linnakangas
On 07/05/18 05:47, Tom Lane wrote: Tatsuo Ishii writes: Does anybody think having in-memory query result cache in core is a good idea? No. Agreed. You could probably write an extension for that, though. I think the planner hook and custom scans give you enough

Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?

2018-05-04 Thread Heikki Linnakangas
On 04/05/18 10:05, Heikki Linnakangas wrote: On 24/04/18 13:57, Kyotaro HORIGUCHI wrote: At Mon, 23 Apr 2018 03:41:47 -0400, Heikki Linnakangas <hlinn...@iki.fi> wrote in <89e33d4f-5c75-0738-3dcb-44c4df59e...@iki.fi> Looking at the patch linked above (https://www.postgresql.or

Re: Considering signal handling in plpython again

2018-05-14 Thread Heikki Linnakangas
On 14/05/18 10:56, Hubert Zhang wrote: For nested SPI case, one option is to turn off the bool variable when entering the SPI function(PLy_spi_prepare, PLy_spi_execute, PLy_cursor etc.) and turn on the bool variable again when exiting the SPI function. Yeah, that seems reasonable. - Heikki

Re: Considering signal handling in plpython again

2018-05-10 Thread Heikki Linnakangas
On 10/05/18 09:32, Hubert Zhang wrote: Hi all, I want to support canceling for a plpython query which may be a busy loop. I found some discussions on pgsql-hackers 2 years ago. Below is the link.

Re: Considering signal handling in plpython again

2018-05-11 Thread Heikki Linnakangas
On 11 May 2018 10:01:56 EEST, Hubert Zhang wrote: >2. Add a flag in hook function to indicate whether to call >Py_AddPendingCall. >This is straightforward.(I prefer it) Yeah, that's what I had in mind, too. A global bool variable that's set when you enter libpython, and

Re: Postgres 11 release notes

2018-05-16 Thread Heikki Linnakangas
On 16/05/18 07:22, Michael Paquier wrote: On Mon, May 14, 2018 at 08:45:44PM -0400, Bruce Momjian wrote: What TLS does is to mix the offered ciphers into the negotiation hash so a man-in-the-middle can't pretend it doesn't support something. Could we do something like that here? I have to

Re: Considering signal handling in plpython again

2018-05-16 Thread Heikki Linnakangas
On 16 May 2018 16:50:24 EEST, Hubert Zhang wrote: >There are remaining two problems > >1. Do we need to consider when to delete the extension hook or it's not >necessary? No, PostgreSQL never unloads shared libraries, so that can be ignored. We used to, and that's what

Re: Memory unit GUC range checks

2018-05-16 Thread Heikki Linnakangas
On 16/05/18 15:19, Heikki Linnakangas wrote: $ postmaster -c track_activity_query_size=1024TB FATAL: invalid value for parameter "track_activity_query_size": "1024TB" HINT: Valid units for this parameter are "kB", "MB", "GB", and "TB&quo

Memory unit GUC range checks

2018-05-16 Thread Heikki Linnakangas
Hi, I played around with the GUC memory units, specifically to test the new GUC_UNIT_BYTES flag (commit 6e7baa32): $ postmaster -c track_activity_query_size=1024kB FATAL: 1048576 is outside the valid range for parameter "track_activity_query_size" (100 .. 102400) $ postmaster -c

Re: Memory unit GUC range checks

2018-05-16 Thread Heikki Linnakangas
On 16/05/18 16:35, Alexander Korotkov wrote: I also think it would be nice to show units in the valid range. I image that if I would see such message at the first time, then I would try to reverse-engineer units from input value representation in the error message. Displaying units in the

Re: An out-of-date comment in nodeIndexonlyscan.c

2018-05-16 Thread Heikki Linnakangas
On 14/05/18 02:15, Thomas Munro wrote: Hello, Since commit cdf91edb (2012), nodeIndexonlyscan.c says: /* * Predicate locks for index-only scans must be acquired at the page * level when the heap is not accessed, since tuple-level predicate

Re: Update backend/lib/README

2018-05-22 Thread Heikki Linnakangas
On 22/05/18 11:44, Ideriha, Takeshi wrote: Hi, When I looked into the dshash.c, I noticed that dshash.c, bipartite_match.c and knapsack.c are not mentioned at README. The other files at src/backend/lib are mentioned. I'm not sure this is an intentional one or just leftovers. Does anyone have

Re: Memory unit GUC range checks

2018-05-23 Thread Heikki Linnakangas
On 17/05/18 00:56, Alexander Korotkov wrote: On Wed, May 16, 2018 at 10:41 PM, Andres Freund wrote: Generally ok, two minor points: diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c index 7cd2d2d80e..93402030f7 100644 {"TB", GUC_UNIT_BLOCKS,

Subplan result caching

2018-05-23 Thread Heikki Linnakangas
still not possible. Thoughts? - Heikki >From d3a738ea6427df908b61a5fdd3ee14a7b78b724a Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas <heikki.linnakan...@iki.fi> Date: Tue, 22 May 2018 08:59:33 +0300 Subject: [PATCH 1/3] Add comment to explain that SubPlan can return its results in two wa

Re: SCRAM with channel binding downgrade attack

2018-05-23 Thread Heikki Linnakangas
On 23/05/18 09:59, Magnus Hagander wrote: With that, a connection would be allowed, if either the server's SSL certificate is verified as with "sslmode=verify-full", *or* SCRAM authentication with channel binding was used. Or perhaps cram it into sslmode,

Re: SCRAM with channel binding downgrade attack

2018-05-23 Thread Heikki Linnakangas
On 22/05/18 11:22, Michael Paquier wrote: On Sat, May 19, 2018 at 09:35:57PM +0900, Michael Paquier wrote: The previous patch has actually problems with servers using "trust", "password" and any protocol which can send directly AUTH_REQ_OK as those could still enforce a downgrade to not use

Re: SCRAM with channel binding downgrade attack

2018-05-25 Thread Heikki Linnakangas
On 25 May 2018 17:44:16 EEST, Robert Haas <robertmh...@gmail.com> wrote: >On Wed, May 23, 2018 at 2:46 AM, Heikki Linnakangas <hlinn...@iki.fi> >wrote: >> We could provide "tls-unique" and "tls-server-end-point" in addition >to >> those,

Re: Redesigning the executor (async, JIT, memory efficiency)

2018-05-25 Thread Heikki Linnakangas
Cool stuff! On 25/05/18 06:35, Andres Freund wrote: For example, this converts this converts TPCH's Q01: tpch_1[26142][1]=# SET client_min_messages ='log'; tpch_1[26142][1]=# SELECT l_returnflag, l_linestatus, sum(l_quantity) AS sum_qty, sum(l_extendedprice) AS

Re: Keeping temporary tables in shared buffers

2018-05-25 Thread Heikki Linnakangas
On 25/05/18 09:25, Asim Praveen wrote: On Thu, May 24, 2018 at 8:19 PM, Tom Lane wrote: So then you have to think about how to transition smoothly between "rel is in local buffers" and "rel is in shared buffers", bearing in mind that ever having the same page in two

Re: XLogWrite uses palloc within a critical section

2018-05-25 Thread Heikki Linnakangas
On 25/05/18 07:45, Kyotaro HORIGUCHI wrote: Hello. I happened to see the following in XLogWrite. ereport(PANIC, (errcode_for_file_access(), errmsg("could not seek in log file %s to offset %u: %m", XLogFileNameP(ThisTimeLineID, openLogSegNo), startoffset)));

Re: pg_control read error message

2018-05-18 Thread Heikki Linnakangas
On 18/05/18 14:32, Magnus Hagander wrote: If pg_control is for some reason empty, we give an error messagfe like: 2018-05-18 13:24:03.342 CEST [19697] PANIC: could not read from control file: Success Which is, uh, wrong -- it's definitely not successful. Obviously this is a state where the

Re: Subplan result caching

2018-05-23 Thread Heikki Linnakangas
On 23/05/18 19:25, Tom Lane wrote: Heikki Linnakangas <hlinn...@iki.fi> writes: I've been working on a patch to add a little cache to SubPlans, to speed up queries with correlated subqueries, where the same subquery is currently executed multiple times with the same parameters. Th

Refactoring query_planner() callback

2018-06-15 Thread Heikki Linnakangas
This still breaks ABI for anything that calls query_planner directly, but it seems somewhat unlikely that there are any such plugins. In v12, we can change the ABI. - Heikki >From 482e86a864f7ae5f4a5ffac09e0d1d21dfd8f762 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Fri, 15 Jun 2018 15:13:41

Re: Supporting tls-server-end-point as SCRAM channel binding for OpenSSL 1.0.0 and 1.0.1

2018-05-29 Thread Heikki Linnakangas
On 29/05/18 17:02, Michael Paquier wrote: Currently, the SCRAM channel binding tls-server-end-point is supported only with OpenSSL 1.0.2 and newer versions as we rely on X509_get_signature_nid to get the certificate signature ID, which is the official way of upstream to get this information as

Re: plans for PostgreSQL 12

2018-06-04 Thread Heikki Linnakangas
On 04/06/18 09:12, Pavel Stehule wrote: 2018-06-04 8:35 GMT+02:00 Simon Riggs : Sounds good. I think this would need to be restricted by operator and datatype, since in general you won't know if the datatype functions need a snapshot or not. Immutable functions for the operators ought to do

Spilling hashed SetOps and aggregates to disk

2018-06-04 Thread Heikki Linnakangas
stgresql.org/message-id/420a0ec7-602c-d406-1e75-1ef7ddc58d83%40iki.fi - Heikki >From 1513a777ca1aa1df57f054ed8d15db9a734adf91 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Thu, 3 May 2018 09:44:13 +0300 Subject: [PATCH 1/2] Optimize memory usage in SetOp executor node. --- src/backe

Re: SCRAM with channel binding downgrade attack

2018-06-02 Thread Heikki Linnakangas
On 28/05/18 15:08, Michael Paquier wrote: On Mon, May 28, 2018 at 12:26:37PM +0300, Heikki Linnakangas wrote: Sounds good. Okay. Done this way as attached. If the backend forces anything else than SCRAM then the client gets an immediate error if channel binding is required, without waiting

Re: pg_config.h.win32 missing a set of flags from pg_config.h.in added in v11 development

2018-06-02 Thread Heikki Linnakangas
On 02/06/18 17:09, Tom Lane wrote: Michael Paquier writes: ... Making HAVE_X509_GET_SIGNATURE_NID a hard requirement bumps the minimal version of OpenSSL supported to 1.0.2, which is something I would not feel much sorry about either like Heikki, as I have heard of many vendors maintaining

Re: [HACKERS] GnuTLS support

2018-06-02 Thread Heikki Linnakangas
On 08/03/18 14:13, Peter Eisentraut wrote: There are two failures in the SSL tests that I cannot explain. The tests are for some rather obscure configurations, so the changed behaviors are not obviously wrong, perhaps legitimate implementation differences. But someone wrote those tests with a

Re: Fix obsolete comment

2018-06-07 Thread Heikki Linnakangas
On 07/06/18 09:43, Amit Langote wrote: A comment in ExecUpdate refers to an argument of ExecConstraints that no longer exists. Attached fixes that, but I'm about over a year too late [1] in sending this patch. Applied, thanks! - Heikki

Pushing down join clauses into subqueries

2018-06-06 Thread Heikki Linnakangas
ery_planner() tell which of them might be useful, e.g. because there are indexes to back them. - Heikki >From d37eb5003b1fcbf25355007821f06c41d23d461c Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Wed, 6 Jun 2018 15:58:49 +0300 Subject: [PATCH 1/2] Push down join quals into lateral subqu

Re: Bug in either collation docs or code

2018-06-06 Thread Heikki Linnakangas
On 05/06/18 07:10, Melanie Plageman wrote: Hi, I noticed what seems like a bug in collation. This query errors out: SELECT (SELECT 'c' COLLATE "de_DE") > (SELECT 'ç' COLLATE "es_ES"); -- error While this query does not: SELECT 'c' COLLATE "de_DE" > (SELECT 'ç' COLLATE "es_ES"); -- no error

Re: SCRAM with channel binding downgrade attack

2018-06-06 Thread Heikki Linnakangas
On 06/06/18 23:20, Peter Eisentraut wrote: Aren't we attacking this on the wrong level? We are here attempting to prevent a SCRAM-SHA-256-PLUS -> SCRAM-SHA-256 downgrade, but we are not preventing a SCRAM-SHA-256-PLUS -> anything-else downgrade. The latest patch does prevent that, too. That

Re: SCRAM with channel binding downgrade attack

2018-06-06 Thread Heikki Linnakangas
On 05/06/18 09:41, Michael Paquier wrote: On Sat, Jun 02, 2018 at 01:08:56PM -0400, Heikki Linnakangas wrote: On 28/05/18 15:08, Michael Paquier wrote: On Mon, May 28, 2018 at 12:26:37PM +0300, Heikki Linnakangas wrote: + printfPQExpBuffer(>errorMessage, +libpq_gettext("channel

Re: SCRAM with channel binding downgrade attack

2018-06-06 Thread Heikki Linnakangas
On 06/06/18 23:31, Peter Eisentraut wrote: On 6/6/18 16:26, Heikki Linnakangas wrote: On 06/06/18 23:20, Peter Eisentraut wrote: Aren't we attacking this on the wrong level? We are here attempting to prevent a SCRAM-SHA-256-PLUS -> SCRAM-SHA-256 downgrade, but we are not preventing a SC

Re: SCRAM with channel binding downgrade attack

2018-05-28 Thread Heikki Linnakangas
On 28/05/18 04:23, Michael Paquier wrote: On Sat, May 26, 2018 at 11:42:38PM +0900, Michael Paquier wrote: On Sat, May 26, 2018 at 09:08:50AM -0400, Bruce Momjian wrote: On Sat, May 26, 2018 at 08:32:20AM +0900, Michael Paquier wrote: OK, I can live with that as well. So we'll go in the

Re: SCRAM with channel binding downgrade attack

2018-05-28 Thread Heikki Linnakangas
On 28/05/18 12:20, Michael Paquier wrote: On Mon, May 28, 2018 at 12:00:33PM +0300, Heikki Linnakangas wrote: That's not a new problem, but it makes the MITM protection fairly pointless, if a fake server can acquire the user's password by simply asking for it. The client will report a failed

Re: Push down Aggregates below joins

2018-06-21 Thread Heikki Linnakangas
On 21/06/18 09:11, Antonin Houska wrote: Tomas Vondra wrote: On 06/20/2018 10:12 PM, Heikki Linnakangas wrote: Currently, the planner always first decides the scan/join order, and adds Group/Agg nodes on top of the joins. Sometimes it would be legal, and beneficial, to perform

Re: BufFileSize() doesn't work on a "shared" BufFiles

2018-05-02 Thread Heikki Linnakangas
On 30/04/18 22:00, Peter Geoghegan wrote: On Mon, Apr 30, 2018 at 10:18 AM, Heikki Linnakangas <hlinn...@iki.fi> wrote: Perhaps that would be OK, if it was properly commented. But it's not actually hard to make BufFileSize() work on shared files, too, so I think we should do that. I

Re: lingering references to V0 calling convention

2018-05-02 Thread Heikki Linnakangas
On 23/04/18 14:17, John Naylor wrote: I noticed one in the SPI docs and tried to look for more. The attached patch is an attempt at removing the remaining references. The original SPI example returned a uint64 as a signed int8 SQL type, so I'm not sure if I handled that correctly. Committed,

Re: Sort performance cliff with small work_mem

2018-05-02 Thread Heikki Linnakangas
On 02/05/18 19:41, Tom Lane wrote: Robert Haas <robertmh...@gmail.com> writes: On Wed, May 2, 2018 at 11:38 AM, Heikki Linnakangas <hlinn...@iki.fi> wrote: To fix, I propose that we change the above so that we always subtract tapeSpace, but if there is less than e.g. 32 kB of memor

BufFileSize() doesn't work on a "shared" BufFiles

2018-04-30 Thread Heikki Linnakangas
ikki >From 6d8c1619c448bc66351eb51393950f898b5c3c5f Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas <heikki.linnakan...@iki.fi> Date: Mon, 30 Apr 2018 20:05:59 +0300 Subject: [PATCH 1/1] Fix some sloppiness in the new BufFileSize() and BufFileAppend() funtions. There were three related issues: * BufFileA

Re: [HACKERS] [BUGS] Bug in Physical Replication Slots (at least 9.5)?

2018-05-04 Thread Heikki Linnakangas
On 24/04/18 13:57, Kyotaro HORIGUCHI wrote: At Mon, 23 Apr 2018 03:41:47 -0400, Heikki Linnakangas <hlinn...@iki.fi> wrote in <89e33d4f-5c75-0738-3dcb-44c4df59e...@iki.fi> Looking at the patch linked above (https://www.postgresql.org/message-id/20171026.190551.208996945.horig

Re: Considering signal handling in plpython again

2018-06-21 Thread Heikki Linnakangas
Hi Hubert, Are you working on this, or should I pick this up? Would be nice to get this done as soon as v12 development begins. - Heikki

Re: location reporting in TAP test failures

2018-06-21 Thread Heikki Linnakangas
On 05/06/18 18:28, Peter Eisentraut wrote: Right now, when a TAP test reports a failure, it looks something like this: # Failed test 'creating a replication slot' # at /../postgresql/src/bin/pg_basebackup/../../../src/test/perl/TestLib.pm line 371. That file location is where we

Re: GiST VACUUM

2018-07-30 Thread Heikki Linnakangas
On 29/07/18 14:47, Andrey Borodin wrote: Fixed both problems. PFA v14. Thanks, took a really quick look at this. The text being added to README is outdated for these latest changes. In second step I still use paloc's memory, but only to store two bitmaps: bitmap of internal pages and bitmap

Re: GiST VACUUM

2018-08-05 Thread Heikki Linnakangas
On 31/07/18 23:06, Andrey Borodin wrote: On a typical GiST index, what's the ratio of leaf vs. internal pages? Perhaps an array would indeed be better. > Typical GiST has around 200 tuples per internal page. I've switched to List since it's more efficient than bitmap. Hmm. A ListCell is 16

Re: Negotiating the SCRAM channel binding type

2018-08-05 Thread Heikki Linnakangas
On 05/08/18 17:11, I wrote: The only negative test there was, was to check for bogus scram_channel_binding option, "scram_channel_binding=not-exists". Yeah, it would be nice to have some, but this commit didn't really change that situation. Sorry, I see now that there was indeed a test for

Re: Negotiating the SCRAM channel binding type

2018-08-05 Thread Heikki Linnakangas
Thanks for the review! On 22/07/18 16:54, Michael Paquier wrote: On Fri, Jul 20, 2018 at 08:05:04PM +0300, Heikki Linnakangas wrote: This removes the scram_channel_binding libpq option altogether, since there is now only one supported channel binding type. This can also be used to switch off

Re: Negotiating the SCRAM channel binding type

2018-08-05 Thread Heikki Linnakangas
On 05/08/18 15:45, Michael Paquier wrote: On Sun, Aug 05, 2018 at 03:30:43PM +0300, Heikki Linnakangas wrote: That test just tested that the scram_channel_binding libpq option works, but I removed the option. I know you wanted to keep it as a feature flag, but as discussed earlier, I don't

Re: Negotiating the SCRAM channel binding type

2018-08-05 Thread Heikki Linnakangas
On 05/08/18 15:08, Michael Paquier wrote: On Sun, Aug 05, 2018 at 02:00:04PM +0300, Heikki Linnakangas wrote: I did some further testing with this, compiling with and without HAVE_BE_TLS_GET_CERTIFICATE_HASH and HAVE_PGTLS_GET_PEER_CERTIFICATE_HASH, and fixed a few combinations that did

Re: [PATCH v18] GSSAPI encryption support

2018-08-05 Thread Heikki Linnakangas
Sorry if this sounds facetious, but: What is the point of this patch? What's the advantage of GSSAPI encryption over SSL? I was hoping to find the answer by reading the documentation changes, but all I can see is "how" to set it up, and nothing about "why". - Heikki

Re: Negotiating the SCRAM channel binding type

2018-08-05 Thread Heikki Linnakangas
On 5 August 2018 19:01:04 EEST, Tom Lane wrote: >Heikki Linnakangas writes: >> Sorry, I see now that there was indeed a test for >> scram_channel_binding='', which meant "no channel binding". That was >> confusing, I assumed '' was the default. > >Ugh, i

Re: Handling better supported channel binding types for SSL implementations

2018-08-06 Thread Heikki Linnakangas
On 28/05/18 06:46, Michael Paquier wrote: On Thu, Mar 08, 2018 at 02:19:55PM -0500, Peter Eisentraut wrote: Moved to next CF along with those other two patches. Attached is a refreshed patch for this thread, which failed to apply after recent changes. This is tied with patches for other SSL

Re: [HACKERS] Optional message to user when terminating/cancelling backend

2018-08-06 Thread Heikki Linnakangas
Has there been any consideration to encodings? What happens if the message contains non-ASCII characters, and the sending backend is connected to database that uses a different encoding than the backend being signaled? - Heikki

Re: Typo in optimizer/README

2018-08-09 Thread Heikki Linnakangas
On 09/08/18 10:22, Tatsuro Yamada wrote: Attached patch for fixing a typo in optimizer/README. s/Partition-wise/Partitionwise/ It is the only place the keyword "Partition-wise" exists, so I created the patch. Thanks, fixed, and also one in a comment in postgres_fdw's regression test. I'm

Re: Negotiating the SCRAM channel binding type

2018-08-08 Thread Heikki Linnakangas
On 07/08/18 17:34, Stephen Frost wrote: * Michael Paquier (mich...@paquier.xyz) wrote: On Tue, Aug 07, 2018 at 02:32:27PM +0530, Robert Haas wrote: On Sun, Aug 5, 2018 at 4:30 PM, Heikki Linnakangas wrote: Well, it'd be useless for users, there is no reason to switch off channel binding

Re: Negotiating the SCRAM channel binding type

2018-08-07 Thread Heikki Linnakangas
On 07/08/18 22:34, Bruce Momjian wrote: On Thu, Jul 12, 2018 at 11:26:30AM +0300, Heikki Linnakangas wrote: On 12/07/18 07:14, Michael Paquier wrote: On Wed, Jul 11, 2018 at 03:01:03PM +0300, Heikki Linnakangas wrote: I started digging into this more closely, and ran into a little problem

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-08-13 Thread Heikki Linnakangas
On 08/08/18 19:19, Heikki Linnakangas wrote: So, with this commit, the various SIGQUIT quickdie handlers are in a better shape. The regular backend's quickdie() handler still calls ereport(), which is not safe, but it's a step in the right direction. And we haven't addressed the original

Re: First steps to being a contributer

2018-08-27 Thread Heikki Linnakangas
On 28/08/18 03:39, Daniel Wood wrote: Having quit Amazon, where I was doing Postgres development, I've started looking at various things I might work on for fun. One thought is to start with something easy like the scalability of GetSnapshotData(). :-) Cool! :-) I recently found it

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 23:29, Fabien COELHO wrote: Hmm. How about we just remove this special case from doCustom(): case CSTATE_START_THROTTLE: // ... if (duration > 0 && st->txn_scheduled > end_time) { st->state = CSTATE_FINISHED; break; } That way, we let the

Re: GiST VACUUM

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 21:27, Andrey Borodin wrote: Hi! 18 июля 2018 г., в 16:02, Heikki Linnakangas написал(а): , but I think it would be better to split this into two patches as follows: 1st patch: Scan the index in physical rather than logical order. No attempt at deleting empty pages yet. 2nd

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-20 Thread Heikki Linnakangas
On 19/07/18 23:04, Nico Williams wrote: On Thu, Jun 22, 2017 at 03:10:31PM -0400, Tom Lane wrote: Andres Freund writes: Or, probably more robust: Simply _exit(2) without further ado, and rely on postmaster to output an appropriate error message. Arguably it's not actually useful to see

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-20 Thread Heikki Linnakangas
On 19/07/18 23:13, Andres Freund wrote: On 2018-07-19 14:54:52 -0500, Nico Williams wrote: On Thu, Jul 19, 2018 at 12:20:53PM -0700, Andres Freund wrote: On 2018-07-19 11:57:25 +0300, Heikki Linnakangas wrote: Ugh. Yeah, in wal_quickdie, and other aux process *_quickdie() handlers, I agree we

Re: pread() and pwrite()

2018-07-20 Thread Heikki Linnakangas
On 20/07/18 01:50, Thomas Munro wrote: An idea for how to handle Windows, in a follow-up patch: add a file src/backend/port/win32/file.c that defines pgwin32_pread() and pgwin32_pwrite(), wrapping WriteFile()/ReadFile() and passing in an "OVERLAPPED" struct with the offset and sets errno on

Re: Negotiating the SCRAM channel binding type

2018-07-20 Thread Heikki Linnakangas
On 12/07/18 16:08, Michael Paquier wrote: On Thu, Jul 12, 2018 at 12:34:51PM +0300, Heikki Linnakangas wrote: Hmm. That is actually in a section called "Default Channel Binding". And the first paragraph says "A default channel binding type agreement process for all SASL applic

Re: [HACKERS][PATCH] Applying PMDK to WAL operations for persistent memory

2018-07-20 Thread Heikki Linnakangas
On 01/03/18 12:40, Heikki Linnakangas wrote: On 16/01/18 15:00, Yoshimi Ichiyanagi wrote: These patches enable to use Persistent Memory Development Kit(PMDK)[1] for reading/writing WAL logs on persistent memory(PMEM). PMEM is next generation storage and it has a number of nice features: fast

Re: [HACKERS] Support for Secure Transport SSL library on macOS as OpenSSL alternative

2018-07-17 Thread Heikki Linnakangas
On 27/06/18 21:57, Daniel Gustafsson wrote: On 27 Jun 2018, at 14:32, Daniel Gustafsson wrote: Attached is an updated patch for supporting the native macOS Secure Transport library, rebased on top of current master. Courtesy of the ever-present Murphy I managed to forget some testcode in

Re: chained transactions

2018-07-17 Thread Heikki Linnakangas
On 01/03/18 05:35, Peter Eisentraut wrote: The SQL standard offers the "chained transactions" feature to address this. The new command variants COMMIT AND CHAIN and ROLLBACK AND CHAIN immediately start a new transaction with the characteristics (isolation level, read/write, deferrable) of the

Re: [HACKERS] WAL logging problem in 9.4.3?

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 16:29, Robert Haas wrote: On Wed, Jul 18, 2018 at 9:06 AM, Michael Paquier wrote: What's wrong with the approach proposed in http://postgr.es/m/55afc302.1060...@iki.fi ? For back-branches that's very invasive so that seems risky to me particularly seeing the low number of

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 16:01, Fabien COELHO wrote: I don't think you want to wait in that situation. I think we should wait at the end only if there some threads still alive, with nothing to do only because of --rate. Yep. The attached version does only the tailing stuff under -R and not all threads were

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 01:43, Fabien COELHO wrote: The more reasonable alternative could be to always last 2 seconds under -T 2, even if the execution can be shorten because there is nothing to do at all, i.e. remove the environment-based condition but keep the sleep. That sounds reasonable. It's a bit

Re: cursors with prepared statements

2018-07-18 Thread Heikki Linnakangas
On 16/07/18 15:56, Peter Eisentraut wrote: On 11.07.18 19:07, Heikki Linnakangas wrote: It's confusing, and risks conflicting with future additions to the standard. ECPG supports the actual standard syntax, with OPEN, right? So this wouldn't be consistent with ECPG, either. It would

Re: [HACKERS] WAL logging problem in 9.4.3?

2018-07-16 Thread Heikki Linnakangas
On 16 July 2018 21:38:39 EEST, Robert Haas wrote: >On Thu, Jul 12, 2018 at 10:12 AM, Heikki Linnakangas >wrote: >> Doesn't have to be a trigger, could be a CHECK constraint, datatype >input >> function, etc. Admittedly, having a datatype input function that >inserts

Re: Make foo=null a warning by default.

2018-07-16 Thread Heikki Linnakangas
On 16/07/18 04:40, David Fetter wrote: Folks, Per a discussion with Andrew Gierth and Vik Fearing, both of whom helped make this happen, please find attached a patch which makes it possible to get SQL standard behavior for "= NULL", which is an error. It's been upgraded to a warning, and can

Re: Runtime partition pruning for MergeAppend

2018-07-19 Thread Heikki Linnakangas
On 18/06/18 13:29, David Rowley wrote: Back in the v11 cycle, there was just not quite enough time to get the MergeAppend run-time partition pruning patch in. I've attached v24 of this patch. The only changes done from v23 [1] are to re-base the patch atop of current master. There's was a bit

Re: pgsql: Fix parallel index and index-only scans to fall back to serial.

2018-07-19 Thread Heikki Linnakangas
On 14/07/18 00:56, Robert Haas wrote: On Fri, Jul 13, 2018 at 2:22 PM, Heikki Linnakangas wrote: I just bumped into this comment, from commit 09529a70bb5, and I can't make sense of it: + /* +* We reach here if the index only scan is not parallel, or if we're

Re: GiST VACUUM

2018-07-19 Thread Heikki Linnakangas
On 19/07/18 13:52, Andrey Borodin wrote: Hi! 19 июля 2018 г., в 1:12, Heikki Linnakangas написал(а): Yeah, please, I think this is the way to go. Here's v11 divided into proposed steps. Thanks, one quick question: /* We should not unlock buffer if we are going

Re: GiST VACUUM

2018-07-19 Thread Heikki Linnakangas
On 19/07/18 14:42, Andrey Borodin wrote: 19.07.2018, 15:20, "Heikki Linnakangas" : On 19/07/18 13:52, Andrey Borodin wrote: Hi! 19 июля 2018 г., в 1:12, Heikki Linnakangas mailto:hlinn...@iki.fi>> написал(а): Yeah, please, I think this

Re: MAP syntax for arrays

2018-07-19 Thread Heikki Linnakangas
On 08/05/18 18:11, Ildar Musin wrote: On 08.05.2018 17:15, Peter Eisentraut wrote: On 5/8/18 09:19, Chapman Flack wrote: On 05/08/2018 08:57 AM, Ildar Musin wrote: select map (pow(2, x) - 1 for x in array[1,2,3,4,5]); I wonder how efficient an implementation would be possible strictly as a

Re: Runtime partition pruning for MergeAppend

2018-07-19 Thread Heikki Linnakangas
On 19/07/18 15:36, David Rowley wrote: I just noticed I managed to miss updating "Append" to "MergeAppend" when copying a comment out of nodeAppend.c into nodeMergeAppend.c. I see you noticed it but accidentally updated the comment in nodeAppend.c. D'oh! Fixed, thanks. - Heikki

Re: [HACKERS] possible self-deadlock window after bad ProcessStartupPacket

2018-07-19 Thread Heikki Linnakangas
On 19/07/18 03:26, Asim R P wrote: On Thu, Jun 22, 2017 at 10:50 AM, Andres Freund wrote: Or, probably more robust: Simply _exit(2) without further ado, and rely on postmaster to output an appropriate error message. Arguably it's not actually useful to see hundreds of "WARNING: terminating

Re: [PATCH] Find additional connection service files in pg_service.conf.d directory

2018-07-19 Thread Heikki Linnakangas
Handy feature! On 01/03/18 20:40, Curt Tilmes wrote: On Thu, Mar 1, 2018 at 1:13 PM, Andres Freund wrote: And within the directory which service file wins will be decided by filesystem internals. That makes me a bit uncomfortable, this very well might not be stable. I think it might not be

Re: [COMMITTERS] pgsql: Give a better error message on invalid hostaddr option.

2018-07-19 Thread Heikki Linnakangas
On 19/07/18 17:07, Robert Haas wrote: On Fri, Jun 9, 2017 at 6:06 AM, Heikki Linnakangas wrote: Give a better error message on invalid hostaddr option. If you accidentally pass a host name in the hostaddr option, e.g. hostaddr=localhost, you get an error like: psql: could not translate host

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-18 Thread Heikki Linnakangas
On 18/07/18 22:56, Fabien COELHO wrote: Hello Heikki, Yep. The attached version does only the tailing stuff under -R and not all threads were stopped on errors, with comments to tell about the why. Hmm. How about we just remove this special case from doCustom(): case

Re: [WIP PATCH] Index scan offset optimisation using visibility map

2018-07-13 Thread Heikki Linnakangas
On 21/05/18 18:43, Michail Nikolaev wrote: Hello everyone. This letter related to “Extended support for index-only-scan” from my previous message in the thread. WIP version of the patch is ready for a while now and I think it is time to resume the work on the feature. BTW, I found a small

Re: PATCH: psql tab completion for SELECT

2018-07-16 Thread Heikki Linnakangas
(trimmed CC list to evade gmail's spam filter, sorry) On 21/03/18 08:51, Edmund Horner wrote: Hi all, I haven't heard anything for a while and so assume you're beavering away on real features. :) I've been dogfooding this patch at work, and I am personally pretty happy with it. I still think

Re: Make foo=null a warning by default.

2018-07-16 Thread Heikki Linnakangas
On 16/07/18 18:10, Tom Lane wrote: Heikki Linnakangas writes: On 16/07/18 04:40, David Fetter wrote: Per a discussion with Andrew Gierth and Vik Fearing, both of whom helped make this happen, please find attached a patch which makes it possible to get SQL standard behavior for "= NULL&qu

Re: [HACKERS] Re: [COMMITTERS] pgsql: Remove pgbench "progress" test pending solution of its timing is (fwd)

2018-07-16 Thread Heikki Linnakangas
On 12/07/18 21:27, Fabien COELHO wrote: For the testing, we just need to make sure that at least one progress report is always printed, if -P is used. Right? Yep. That is the first condition above the last_report is set to thread_start meaning that there has been no report. So where does the

Re: GiST VACUUM

2018-07-18 Thread Heikki Linnakangas
On 17/07/18 21:41, Andrey Borodin wrote: I was checking WAL replay of new scheme to log page deletes and found a bug there (incorrect value of deleted downlink in WAL record). Here's fixed patch v10. Also I've added support to WAL identification for new record, done some improvements to

Re: Implement predicate propagation for non-equivalence clauses

2018-09-05 Thread Heikki Linnakangas
On 05/09/18 09:34, Richard Guo wrote: Hi, As we know, current planner will generate additional restriction clauses from equivalence clauses. This will generally lower the total cost because some of tuples may be filtered out before joins. In this patch, we are trying to do the similar

Re: Fix comments of IndexInfo

2018-08-30 Thread Heikki Linnakangas
On 30/08/18 07:48, Yugo Nagata wrote: Attached is a patch to fix comments of IndexInfo. ii_KeyAttrNumbers was renamed to ii_IndexAttrNumbers and ii_Am was added but these are not reflected to the comment. Applied, thanks! - Heikki

Re: Excessive PostmasterIsAlive calls slow down WAL redo

2018-07-10 Thread Heikki Linnakangas
rom 9418ed472d113a80bf0fbb209efb0835767a5c50 Mon Sep 17 00:00:00 2001 From: Heikki Linnakangas Date: Tue, 10 Jul 2018 14:31:48 +0300 Subject: [PATCH 1/1] Use signals for postmaster death on Linux. Linux provides a way to ask for a signal when your parent process dies. Use that to make PostmasterIsAlive() very cheap. Author: Tho

Re: Test for trigger condition accessing system attributes

2018-07-10 Thread Heikki Linnakangas
On 10/05/18 11:58, Ashutosh Bapat wrote: Hi, I was investigating the cases when the system attributes are accessed beyond the scans. After investigating set_plan_references(), I thought that we never access system attributes beyond scans. This lead me to assume that EEOP_INNER/OUTER_SYSVAR are

Re: [HACKERS] WAL logging problem in 9.4.3?

2018-07-10 Thread Heikki Linnakangas
Thanks for picking this up! (I hope this gets through the email filters this time, sending a shell script seems to be difficult. I also trimmed the CC list, if that helps.) On 04/07/18 07:59, Michael Paquier wrote: Hence I propose the patch attached which disables the TRUNCATE and COPY

Re: Optimze usage of immutable functions as relation

2018-07-10 Thread Heikki Linnakangas
On 16/05/18 13:47, Aleksandr Parfenov wrote: Hello, I reworked a patch to make more stable in different cases. I decided to use simplify_function instead of eval_const_expression to prevent inlining of the function. The only possible outputs of the simplify_function are Const node and NULL.

Re: GiST VACUUM

2018-07-10 Thread Heikki Linnakangas
I'm now looking at the first patch in this series, to allow completely empty GiST pages to be recycled. I've got some questions: --- a/src/backend/access/gist/gist.c +++ b/src/backend/access/gist/gist.c @@ -700,6 +700,13 @@ gistdoinsert(Relation r, IndexTuple itup, Size freespace, GISTSTATE

Re: Excessive PostmasterIsAlive calls slow down WAL redo

2018-07-11 Thread Heikki Linnakangas
On 11/07/18 04:16, Thomas Munro wrote: On Tue, Jul 10, 2018 at 11:39 PM, Heikki Linnakangas wrote: I don't have a FreeBSD machine at hand, so I didn't try fixing that patch. I updated the FreeBSD version to use the header test approach you showed, and pushed that too. FWIW the build farm

Negotiating the SCRAM channel binding type

2018-07-11 Thread Heikki Linnakangas
Currently, there is no negotiation of the channel binding type between client and server. The server advertises that it supports channel binding, or not, and the client decides what channel binding to use. If the server doesn't support the binding type that the client chose, authentication

Re: [HACKERS] GnuTLS support

2018-07-11 Thread Heikki Linnakangas
On 05/06/18 00:44, Peter Eisentraut wrote: On 6/2/18 16:50, Heikki Linnakangas wrote: On 08/03/18 14:13, Peter Eisentraut wrote: There are two failures in the SSL tests that I cannot explain. The tests are for some rather obscure configurations, so the changed behaviors are not obviously

  1   2   3   4   5   6   7   8   9   10   >