Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Wed, Apr 4, 2018 at 2:44 PM, Thomas Munro wrote: > On Wed, Apr 4, 2018 at 2:14 PM, Bruce Momjian wrote: >> Uh, are you sure it fixes our use-case? From the email description it >> sounded like it only reported fsync errors for every open file

Re: [HACKERS] Runtime Partition Pruning

Hi David, On Wed, Apr 4, 2018 at 7:57 AM, David Rowley wrote: > On 4 April 2018 at 05:50, Beena Emerson wrote: >> With Amit's v46 patch, the following query in partition_prune was >> crashing during make check. >> explain (analyze, costs

Re: [HACKERS] path toward faster partition pruning

On 4 April 2018 at 16:00, Tom Lane wrote: > David Rowley writes: >> It's true that the const simplification code will generally rewrite >> most NOT(clause) to use the negator operator, but if the operator does >> not have a negator it can't do

Re: [HACKERS] path toward faster partition pruning

David Rowley writes: > It's true that the const simplification code will generally rewrite > most NOT(clause) to use the negator operator, but if the operator does > not have a negator it can't do this. > ... > At the moment pruning does not work for this case at

Re: WIP: Covering + unique indexes.

On Tue, Apr 3, 2018 at 7:02 AM, Alexander Korotkov wrote: > Great, I'm looking forward your feedback. I took a look at V11 (0001-Covering-core-v11.patch, 0002-Covering-btree-v11.patch, 0003-Covering-amcheck-v11.patch, 0004-Covering-natts-v11.patch) today. * What's a

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

On Wed, Apr 4, 2018 at 4:31 AM, Andres Freund wrote: > On 2018-03-06 19:57:03 +0530, Amit Kapila wrote: >> On Tue, Mar 6, 2018 at 4:53 AM, Andres Freund wrote: >> > Hi, >> > >> >> diff --git a/src/backend/executor/nodeLockRows.c >> >>

Re: [PATCH] Logical decoding of TRUNCATE

Here is an updated patch that addresses some of your concerns. I've split it up into the decoding support and the pgoutput replication support. The problem I see now is that when we WAL-log a truncate, we include all the relids in one record. That seems useful. But during decoding we split

Re: [HACKERS] Runtime Partition Pruning

On 4 April 2018 at 05:50, Beena Emerson wrote: > With Amit's v46 patch, the following query in partition_prune was > crashing during make check. > explain (analyze, costs off, summary off, timing off) execute ab_q1 (2, 2, 3); Hi Beena, Thanks for looking. Does it work

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Wed, Apr 4, 2018 at 01:54:50PM +1200, Thomas Munro wrote: > On Wed, Apr 4, 2018 at 12:56 PM, Bruce Momjian wrote: > > There has been a lot of focus in this thread on the workflow: > > > > write() -> blocks remain in kernel memory -> fsync() -> panic? > > > > But what

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Wed, Apr 4, 2018 at 12:56 PM, Bruce Momjian wrote: > There has been a lot of focus in this thread on the workflow: > > write() -> blocks remain in kernel memory -> fsync() -> panic? > > But what happens in this workflow: > > write() -> kernel syncs blocks to

Re: [HACKERS] path toward faster partition pruning

On 4 April 2018 at 11:22, David Rowley wrote: > On 4 April 2018 at 09:47, David Rowley wrote: >> I think it would be better to just have special handling in >> get_matching_list_bound so that it knows it's performing <> >> elimination.

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Tue, Apr 3, 2018 at 05:47:01PM -0400, Robert Haas wrote: > Well, in PostgreSQL, we have a background process called the > checkpointer which is the process that normally does all of the > fsync() calls but only a subset of the write() calls. The > checkpointer does not, however, necessarily

Re: Running Installcheck remotely

On Tue, Apr 03, 2018 at 09:12:48AM -0700, Kapil Sharma wrote: > So this means that the host running the test should have capability to SSH > to the DB Instance host ? That's not what it means as the whole test will run through libpq. Please make sure that the files used for the tests match the

Re: open/lseek overhead with many partitions (including with "faster partitioning pruning")

On 4 April 2018 at 12:31, Amit Langote wrote: > On 2018/04/04 9:27, David Rowley wrote: >> Yeah, this will no doubt be due to the fact that we still build >> RelOptInfos in the planner for all partitions, so we're still doing >> get_relation_info() and

Re: open/lseek overhead with many partitions (including with "faster partitioning pruning")

On 2018/04/04 9:27, David Rowley wrote: > On 4 April 2018 at 07:46, Justin Pryzby wrote: >> TLDR: even with "faster pruning" patch, pg11dev open()+lseek() every file >> backing every table being queried, even for those partitions eventually >> "excluded". > > Yeah, this

Re: open/lseek overhead with many partitions (including with "faster partitioning pruning")

Hi Justin. On 2018/04/04 4:46, Justin Pryzby wrote: > TLDR: even with "faster pruning" patch, pg11dev open()+lseek() every file > backing every table being queried, even for those partitions eventually > "excluded". That's expected. The faster pruning patch doesn't change the behavior with

Re: open/lseek overhead with many partitions (including with "faster partitioning pruning")

On 4 April 2018 at 07:46, Justin Pryzby wrote: > TLDR: even with "faster pruning" patch, pg11dev open()+lseek() every file > backing every table being queried, even for those partitions eventually > "excluded". Yeah, this will no doubt be due to the fact that we still build

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Tue, Apr 3, 2018 at 1:29 PM, Thomas Munro wrote: > Interestingly, there don't seem to be many operating systems that can > report ENOSPC from fsync(), based on a quick scan through some > documentation: > > POSIX, AIX, HP-UX, FreeBSD, OpenBSD, NetBSD: no >

Re: [HACKERS] path toward faster partition pruning

On 4 April 2018 at 09:47, David Rowley wrote: > On 4 April 2018 at 00:02, Amit Langote wrote: >> But actually, the presence of only Params in the pruning steps should >> result in the pruning not being invoked at all (at least for the

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

On 2018-03-06 19:57:03 +0530, Amit Kapila wrote: > On Tue, Mar 6, 2018 at 4:53 AM, Andres Freund wrote: > > Hi, > > > >> diff --git a/src/backend/executor/nodeLockRows.c > >> b/src/backend/executor/nodeLockRows.c > >> index 7961b4be6a..b07b7092de 100644 > >> ---

Re: Online enabling of checksums

On 04/03/2018 02:05 PM, Magnus Hagander wrote: > On Sun, Apr 1, 2018 at 2:04 PM, Magnus Hagander > wrote: > > On Sat, Mar 31, 2018 at 5:38 PM, Tomas Vondra > > >

Re: Optimize Arm64 crc32c implementation in Postgresql

On Wed, Apr 4, 2018 at 4:38 AM, Heikki Linnakangas wrote: > [armv8-crc32c-2.patch] This tests OK on my Debian buster aarch64 system (the machine that runs "eelpout" in the buildfarm), configure tests as expected and I confirmed that pg_comp_crc32c_armv8 is reached at runtime. I

Re: [HACKERS] logical decoding of two-phase transactions

On Wed, Apr 4, 2018 at 12:25 AM, Tom Lane wrote: > Tomas Vondra writes: >> On 04/03/2018 04:37 PM, Alvaro Herrera wrote: >>> Tomas Vondra wrote: Yes, that is a good point actually - we need to test that replication between PG10 and PG11

Re: Foreign keys and partitioned tables

Peter Eisentraut wrote: > > 0002 is a fixup for a bug in the row triggers patch: I had a restriction > > earlier that triggers declared internal were not cloned, and I seem to > > have lost it in rebase. Reinstate it. > > Hmm, doesn't cause any test changes? Here's a test case: create table t

Re: [HACKERS] path toward faster partition pruning

On 4 April 2018 at 00:02, Amit Langote wrote: > But actually, the presence of only Params in the pruning steps should > result in the pruning not being invoked at all (at least for the static > pruning case), thus selecting all partitions containing non-null data.

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Tue, Apr 3, 2018 at 6:35 AM, Anthony Iliopoulos wrote: >> Like other people here, I think this is 100% unreasonable, starting >> with "the dirty pages which cannot been written out are practically >> thrown away". Who decided that was OK, and on the basis of what >>

Re: pgsql: Validate page level checksums in base backups

On 4/3/18 4:48 PM, Michael Banck wrote: Attached is a patch which does that hopefully: 1. creates two user tables, one large enough for at least 6 blocks (around 360kb), the other just one block. 2. stops the cluster before scribbling over its data and starts it afterwards. 3. uses the

Re: pgsql: Validate page level checksums in base backups

Hi, On Tue, Apr 03, 2018 at 08:48:08PM +0200, Magnus Hagander wrote: > On Tue, Apr 3, 2018 at 8:29 PM, Tom Lane wrote: > I'd bet a good lunch that nondefault BLCKSZ would break it, as well, > > since the way in which the corruption is induced is just guessing > > as to where

Re: Foreign keys and partitioned tables

Alvaro Herrera wrote: > While adding some more tests for the "action" part (i.e. updates and > deletes on the referenced table) I came across a bug that was causing > the server to crash ... but it's actually a preexisting bug in an > assert. The fix is in 0001. Yeah, it's a live bug that only

open/lseek overhead with many partitions (including with "faster partitioning pruning")

TLDR: even with "faster pruning" patch, pg11dev open()+lseek() every file backing every table being queried, even for those partitions eventually "excluded". One of our customers (with the largest number of child tables, currently of relkind='r') takes a long time to plan query, and I wondered

Re: Foreign keys and partitioned tables

While adding some more tests for the "action" part (i.e. updates and deletes on the referenced table) I came across a bug that was causing the server to crash ... but it's actually a preexisting bug in an assert. The fix is in 0001. 0002 I already posted: don't clone row triggers that are

Re: Optimize Arm64 crc32c implementation in Postgresql

> On 03 Apr 2018, at 18:38, Heikki Linnakangas wrote: > > On 03/04/18 19:09, Andres Freund wrote: >> Hi, >> On 2018-04-03 19:05:19 +0300, Heikki Linnakangas wrote: >>> On 01/04/18 20:32, Andres Freund wrote: On 2018-03-06 02:44:35 +0800, Heikki Linnakangas wrote: > * I

Re: pgsql: Validate page level checksums in base backups

Magnus Hagander writes: > On Tue, Apr 3, 2018 at 8:29 PM, Tom Lane wrote: >> It's scribbling on the source cluster's disk files and assuming that that >> translates one-for-one to what gets sent to the slave server --- but what >> if some of the blocks

Re: pgsql: Validate page level checksums in base backups

On Tue, Apr 3, 2018 at 8:29 PM, Tom Lane wrote: > Magnus Hagander writes: > > Yeah, there's clearly a second problem here. > > I think this test script is broken in many ways. > > It's scribbling on the source cluster's disk files and assuming that that

Re: pgsql: Validate page level checksums in base backups

On Tue, Apr 3, 2018 at 11:29 AM, Tom Lane wrote: > Also, scribbling on tables as sensitive as pg_class is just asking for > trouble IMO. I don't see anything in this test, for example, that > prevents autovacuum from running and causing a PANIC before the test > can complete.

Re: pgsql: Validate page level checksums in base backups

Magnus Hagander writes: > Yeah, there's clearly a second problem here. I think this test script is broken in many ways. It's scribbling on the source cluster's disk files and assuming that that translates one-for-one to what gets sent to the slave server --- but what if

Re: Comment update in BuildTupleFromCStrings()

On Fri, Mar 23, 2018 at 03:00:37PM +0530, Ashutosh Bapat wrote: > Hi, > BuildTupleFromCStrings() has comment "/* Call the "in" function for > each non-dropped attribute */". It then calls the in function even > when it's going to set that attribute to NULL. > 1189 if

Re: [HACKERS] Runtime Partition Pruning

Hello, On Tue, Apr 3, 2018 at 11:14 PM, Jesper Pedersen wrote: > Hi David, > > On 03/31/2018 09:52 AM, David Rowley wrote: >> >> I've attached a new version of the patch. I'm now at v18 after having >> some versions of the patch that I didn't release which were based

Re: Optimize Arm64 crc32c implementation in Postgresql

Heikki Linnakangas writes: > I was just about to commit this, when I started to wonder: Do we need to > worry about alignment? As the patch stands, it will merrily do unaligned > 8-byte loads. Is that OK on ARM? It seems to work on the system I've > been testing on, but I

Re: [HACKERS] Runtime Partition Pruning

Hi David, On 03/31/2018 09:52 AM, David Rowley wrote: I've attached a new version of the patch. I'm now at v18 after having some versions of the patch that I didn't release which were based on various versions of Amit's faster partition pruning patch. Thank you for the updated patch set !

Re: BRIN FSM vacuuming questions

I wrote: > [ assorted complaining about BRIN FSM management ] Here's a proposed patch for this. Changes: * Do RecordPageWithFreeSpace unconditionally in brin_page_cleanup, and do FreeSpaceMapVacuum unconditionally in brin_vacuum_scan. Because of the FSM's roundoff behavior, the old

Re: BUG #15122: can't import data if table has a constraint with a function calling another function

On Tue, Mar 20, 2018 at 12:11:45PM +, Andrew Gierth wrote: > > "Asier" == Asier Lostalé writes: > > Asier> Thanks Andrew for your quick response and clear explanation. > > Asier> Can I understand from your explanation this is not considered as > Asier> a

Re: pgsql: Validate page level checksums in base backups

On Tue, Apr 3, 2018 at 7:13 PM, Andres Freund wrote: > On 2018-04-03 11:52:26 +, Magnus Hagander wrote: > > Validate page level checksums in base backups > > > > When base backups are run over the replication protocol (for example > > using pg_basebackup), verify the

Re: pgsql: Validate page level checksums in base backups

On 2018-04-03 11:52:26 +, Magnus Hagander wrote: > Validate page level checksums in base backups > > When base backups are run over the replication protocol (for example > using pg_basebackup), verify the checksums of all data blocks if > checksums are enabled. If checksum failures are

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Tue, Apr 03, 2018 at 03:37:30PM +0100, Greg Stark wrote: > On 3 April 2018 at 14:36, Anthony Iliopoulos wrote: > > > If EIO persists between invocations until explicitly cleared, a process > > cannot possibly make any decision as to if it should clear the error > > I

Re: Prefix operator for text and spgist support

Thank you, pushed with some editorization and renaming text_startswith to starts_with Ildus Kurbangaliev wrote: On Fri, 23 Mar 2018 11:45:33 +0300 Alexander Korotkov wrote: On Thu, Mar 22, 2018 at 7:09 PM, Teodor Sigaev wrote: Patch looks

Re: Optimize Arm64 crc32c implementation in Postgresql

Hi, On 2018-04-03 19:38:42 +0300, Heikki Linnakangas wrote: > I was just about to commit this, when I started to wonder: Do we need to > worry about alignment? As the patch stands, it will merrily do unaligned > 8-byte loads. Is that OK on ARM? It seems to work on the system I've been > testing

Re: Optimize Arm64 crc32c implementation in Postgresql

On 03/04/18 19:09, Andres Freund wrote: Hi, On 2018-04-03 19:05:19 +0300, Heikki Linnakangas wrote: On 01/04/18 20:32, Andres Freund wrote: On 2018-03-06 02:44:35 +0800, Heikki Linnakangas wrote: * I tested this on Linux, with gcc and clang, on an ARM64 virtual machine that I had available

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

On 2018-04-03 09:56:24 -0400, Tom Lane wrote: > Heikki Linnakangas writes: > > But let's go back to why we're considering this. The idea was to > > optimize this block: > > ... > > One trick that we could do is to replace that with a 128-bit atomic > > compare-and-swap

Re: Running Installcheck remotely

So this means that the host running the test should have capability to SSH to the DB Instance host ? Thanks, Kapil. On Sat, Mar 24, 2018 at 7:21 AM, Tom Lane wrote: > Kapil Sharma writes: > > Is it possible to run installcheck (pg_regress) tests

Re: Optimize Arm64 crc32c implementation in Postgresql

Hi, On 2018-04-03 19:05:19 +0300, Heikki Linnakangas wrote: > On 01/04/18 20:32, Andres Freund wrote: > > On 2018-03-06 02:44:35 +0800, Heikki Linnakangas wrote: > > > * I tested this on Linux, with gcc and clang, on an ARM64 virtual machine > > > that I had available (not an emulator, but a VM

Re: Optimize Arm64 crc32c implementation in Postgresql

On 01/04/18 20:32, Andres Freund wrote: On 2018-03-06 02:44:35 +0800, Heikki Linnakangas wrote: * I tested this on Linux, with gcc and clang, on an ARM64 virtual machine that I had available (not an emulator, but a VM on a shared ARM64 server). Have you seen actual postgres performance

Re: [HACKERS] GUC for cleanup indexes threshold.

Sorry for the late response. On Tue, Mar 27, 2018 at 8:01 PM, Alexander Korotkov wrote: > On Tue, Mar 27, 2018 at 11:55 AM, Masahiko Sawada > wrote: >> >> On Thu, Mar 22, 2018 at 9:24 PM, Alexander Korotkov >> wrote:

Re: pgsql: New files for MERGE

Hi, On 2018-04-03 09:24:12 +, Simon Riggs wrote: > New files for MERGE > src/backend/executor/nodeMerge.c | 575 +++ > src/backend/parser/parse_merge.c | 660 > src/include/executor/nodeMerge.h | 22 + >

Re: [HACKERS] logical decoding of two-phase transactions

Hi Tomas, >> Unfortunately, this does segfault for me in `make check` almost immediately. Try This is due to the new ERROR handling code that I added today for the lock/unlock APIs. Will fix. >> Also, current value for LOGICALREP_IS_COMMIT is 1, but previous code expected flags to be zero, so

Re: [HACKERS] logical decoding of two-phase transactions

FWIW, a couple of additional comments based on eyeballing the diffs: 1) twophase.c - I think this comment is slightly inaccurate: /* * Coordinate with logical decoding backends that may be already * decoding this prepared transaction. When aborting a transaction, * we need to wait

Re: [HACKERS] logical decoding of two-phase transactions

Tomas Vondra writes: > On 04/03/2018 04:37 PM, Alvaro Herrera wrote: >> Tomas Vondra wrote: >>> Yes, that is a good point actually - we need to test that replication >>> between PG10 and PG11 works correctly, i.e. that the protocol version is >>> correctly

Re: [PATCH] Verify Checksums during Basebackups

On Tue, Apr 3, 2018 at 4:32 PM, Tom Lane wrote: > I wrote: > > Magnus Hagander writes: > >> Unless.. %ld is the wrong thing to print: > >> static int64 total_checksum_failures; > >> We should perhaps be using something other than %ld to print that? > > >

Re: jsonb nesting level edge case

On Sun, Mar 18, 2018 at 09:42:14PM +0100, Dmitry Dolgov wrote: > Hi, > > I've just realized, that looks like there is one edge-case in the current > jsonb > implementation, that can be quite confusing, and I couldn't find any related > discussion about it. From what I see there is no limit for

Re: [HACKERS] logical decoding of two-phase transactions

On 04/03/2018 04:37 PM, Alvaro Herrera wrote: > Tomas Vondra wrote: > >> Yes, that is a good point actually - we need to test that replication >> between PG10 and PG11 works correctly, i.e. that the protocol version is >> correctly negotiated, and features are disabled/enabled accordingly etc. >

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 3 April 2018 at 14:36, Anthony Iliopoulos wrote: > If EIO persists between invocations until explicitly cleared, a process > cannot possibly make any decision as to if it should clear the error I still don't understand what "clear the error" means here. The writes still

Re: [HACKERS] logical decoding of two-phase transactions

Tomas Vondra wrote: > Yes, that is a good point actually - we need to test that replication > between PG10 and PG11 works correctly, i.e. that the protocol version is > correctly negotiated, and features are disabled/enabled accordingly etc. Maybe it'd be good to have a buildfarm animal to

Re: [HACKERS] logical decoding of two-phase transactions

On 04/03/2018 04:07 PM, Stas Kelvich wrote: > > >> On 3 Apr 2018, at 16:56, Tomas Vondra wrote: >> >> >> So I think we need a subscription parameter to enable/disable this, >> defaulting to 'disabled’. > > +1 > > Also, current value for LOGICALREP_IS_COMMIT is

Re: [PATCH] Verify Checksums during Basebackups

I wrote: > Magnus Hagander writes: >> Unless.. %ld is the wrong thing to print: >> static int64 total_checksum_failures; >> We should perhaps be using something other than %ld to print that? > INT64_FORMAT. BTW, don't just stick INT64_FORMAT into the

Re: [PATCH] Verify Checksums during Basebackups

Magnus Hagander writes: > Unless.. %ld is the wrong thing to print: > static int64 total_checksum_failures; > We should perhaps be using something other than %ld to print that? INT64_FORMAT. regards, tom lane

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 3 April 2018 at 10:54, Robert Haas wrote: > I think it's always unreasonable to > throw away the user's data. Well, we do that. If a txn aborts, all writes in the txn are discarded. I think that's perfectly reasonable. Though we also promise an all or nothing

Re: [PATCH] Verify Checksums during Basebackups

On Tue, Apr 3, 2018 at 4:25 PM, Tom Lane wrote: > Magnus Hagander writes: > > Seems the tests are failing on prairiedog: > > https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl? > nm=prairiedog=2018-04-03%2012%3A15%3A27=pg_basebackup-check > > I

Re: [PATCH] Verify Checksums during Basebackups

Magnus Hagander writes: > Seems the tests are failing on prairiedog: > https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=prairiedog=2018-04-03%2012%3A15%3A27=pg_basebackup-check > I don't have time to dive in right now, but that seems interesting -- it's >

Re: Vacuum: allow usage of more than 1GB of work mem

On Tue, Apr 3, 2018 at 11:09 AM, Claudio Freire wrote: > On Tue, Apr 3, 2018 at 11:06 AM, Claudio Freire > wrote: >> I didn't receive your comment, I just saw it. Nevertheless, I rebased the >> patches a while ago just because I noticed they

Re: new function for tsquery creartion

On Tue, 03 Apr 2018 14:28:37 +0300 Dmitry Ivanov wrote: > I'm sorry, I totally forgot to fix a few more things, the patch is > attached below. The patch looks good to me except two things. I'm not sure about the different result for these queries: SELECT

Re: Vacuum: allow usage of more than 1GB of work mem

On Tue, Apr 3, 2018 at 11:06 AM, Claudio Freire wrote: > I didn't receive your comment, I just saw it. Nevertheless, I rebased the > patches a while ago just because I noticed they didn't apply anymore in > cputube, and they still seem to apply. Sorry, that is false.

Re: [HACKERS] logical decoding of two-phase transactions

> On 3 Apr 2018, at 16:56, Tomas Vondra wrote: > > > So I think we need a subscription parameter to enable/disable this, > defaulting to 'disabled’. +1 Also, current value for LOGICALREP_IS_COMMIT is 1, but previous code expected flags to be zero, so this way

Re: Vacuum: allow usage of more than 1GB of work mem

I didn't receive your comment, I just saw it. Nevertheless, I rebased the patches a while ago just because I noticed they didn't apply anymore in cputube, and they still seem to apply. Patch number 2 was committed a long while ago, that's why it's missing. It was a simple patch, it landed

Re: [PATCH] Verify Checksums during Basebackups

On Tue, Apr 3, 2018 at 4:00 PM, Michael Banck wrote: > Hi Magnus, > > Am 03.04.2018 13:59 schrieb Magnus Hagander : > > And of course I forgot that particular part in the first push, so I've > pushed it as a separate commit. > > > Many thanks for

Re: WIP: Covering + unique indexes.

On Tue, Apr 3, 2018 at 7:02 AM, Peter Geoghegan wrote: > On Mon, Apr 2, 2018 at 4:27 PM, Alexander Korotkov > wrote: > > I thought abut that another time and I decided that it would be safer > > to use 13th bit in index tuple flags. There are already

Re: [PATCH] Verify Checksums during Basebackups

Hi Magnus,Am 03.04.2018 13:59 schrieb Magnus Hagander :And of course I forgot that particular part in the first push, so I've pushed it as a separate commit. Many thanks for cleaning up the patch and committing it!Michael-- Michael BanckProjektleiter / Senior BeraterTel.: +49

Re: Transform for pl/perl

On 3/15/18 03:46, Pavel Stehule wrote: > It looks well > > the patch has tests and doc, > there are not any warnings or compilation issues > all tests passed > > I'll mark this patch as ready for commiter committed -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL

Re: [HACKERS] logical decoding of two-phase transactions

On 04/03/2018 12:40 PM, Nikhil Sontakke wrote: > Hi, > >>> It's certainly a nice future goal to have it all happen automatically, >>> but we don't know what the plugin will do. >> >> No, fighting too complicated APIs is not unreasonable. And we've found >> an alternative. >> > > PFA, latest

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

Heikki Linnakangas writes: > But let's go back to why we're considering this. The idea was to > optimize this block: > ... > One trick that we could do is to replace that with a 128-bit atomic > compare-and-swap instruction. Modern 64-bit Intel systems have that, > it's called

Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently

On Tue, Apr 3, 2018 at 10:19 AM, Claudio Freire wrote: > On Thu, Mar 29, 2018 at 2:09 PM, Tom Lane wrote: >> Claudio Freire writes: >>> On Wed, Mar 28, 2018 at 6:54 PM, Tom Lane wrote: After 0001,

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Tue, Apr 03, 2018 at 12:26:05PM +0100, Greg Stark wrote: > On 3 April 2018 at 11:35, Anthony Iliopoulos wrote: > > Hi Robert, > > > > Fully agree, and the errseq_t fixes have dealt exactly with the issue > > of making sure that the error is reported to all file descriptors

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

On Thu, Mar 29, 2018 at 5:18 PM, Tom Lane wrote: > Tomas Vondra writes: >> If each WAL record has xl_curr, then we know to which position the >> record belongs (after verifying the checksum). And we do know the size >> of each WAL record, so we

Re: [HACKERS] [PATCH] Vacuum: Update FSM more frequently

On Thu, Mar 29, 2018 at 7:55 PM, Tom Lane wrote: > I wrote: >> I have to go do something >> else right now, but I'll take a closer look at 0004 later. > > OK, so after studying 0004, it seems to me that we could do it more > simply as attached; that is, move the

Re: [HACKERS] Add support for tuple routing to foreign partitions

(2018/04/03 13:59), Amit Langote wrote: On 2018/04/02 21:29, Etsuro Fujita wrote: (2018/04/02 18:49), Amit Langote wrote: I looked at the new patch. It looks good overall, although I have one question -- IIUC, BeginForeignInsert() performs actions that are equivalent of performing

Re: [HACKERS] Add support for tuple routing to foreign partitions

(2018/04/03 13:32), Amit Langote wrote: On 2018/04/02 21:26, Etsuro Fujita wrote: We wouldn't need that for foreign partitions (When DO NOTHING with an inference specification or DO UPDATE on a partitioned table containing foreign partitions, the planner would throw an error before we get to

Re: Online enabling of checksums

On Sun, Apr 1, 2018 at 2:04 PM, Magnus Hagander wrote: > On Sat, Mar 31, 2018 at 5:38 PM, Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > >> On 03/31/2018 05:05 PM, Magnus Hagander wrote: >> > On Sat, Mar 31, 2018 at 4:21 PM, Tomas Vondra >> >

Re: [PATCH] Verify Checksums during Basebackups

On Tue, Apr 3, 2018 at 1:52 PM, Magnus Hagander wrote: > > > On Mon, Apr 2, 2018 at 2:48 PM, Michael Banck > wrote: > >> Hi! >> >> On Sun, Apr 01, 2018 at 05:27:21PM +0200, Magnus Hagander wrote: >> > It might be a micro-optimisation, but ISTM

Re: [PROPOSAL] Shared Ispell dictionaries

On Thu, Mar 29, 2018 at 02:03:07AM +0300, Arthur Zakirov wrote: > Here is the new version of the patch. Please find the attached new version of the patch. I removed refcnt because it is useless, it doesn't guarantee that a hash table entry will be removed. I fixed a bug, dsm_unpin_segment() can

Re: [PATCH] Verify Checksums during Basebackups

On Mon, Apr 2, 2018 at 2:48 PM, Michael Banck wrote: > Hi! > > On Sun, Apr 01, 2018 at 05:27:21PM +0200, Magnus Hagander wrote: > > It might be a micro-optimisation, but ISTM that we shouldn't do the > > basename(palloc(fn)) in is_heap_file() unless we actually need it

Re: new function for tsquery creartion

The code in its current state looks messy and way too complicated; there're lots of interleaving code branches. Thus, I decided to split gettoken_query() into three independent tokenizers for phrase, web and original (to_tsquery()) syntaxes. Documentation is included. Any feedback is very

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 3 April 2018 at 11:35, Anthony Iliopoulos wrote: > Hi Robert, > > Fully agree, and the errseq_t fixes have dealt exactly with the issue > of making sure that the error is reported to all file descriptors that > *happen to be open at the time of error*. But I think one would

Re: new function for tsquery creartion

Hi everyone, The code in its current state looks messy and way too complicated; there're lots of interleaving code branches. Thus, I decided to split gettoken_query() into three independent tokenizers for phrase, web and original (to_tsquery()) syntaxes. Documentation is included. Any

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

Hi Robert, On Mon, Apr 02, 2018 at 10:54:26PM -0400, Robert Haas wrote: > On Mon, Apr 2, 2018 at 2:53 PM, Anthony Iliopoulos wrote: > > Given precisely that the dirty pages which cannot been written-out are > > practically thrown away, the semantics of fsync() (after the 4.13

Re: Commit 4dba331cb3 broke ATTACH PARTITION behaviour.

On 2018/04/03 14:45, Kyotaro HORIGUCHI wrote: > Hello. > > At Mon, 2 Apr 2018 16:11:12 -0300, Alvaro Herrera > wrote: >> Why do we need AccessExclusiveLock on all children of a relation that we >> want to scan to search for rows not satisfying the constraint? I think

Re: Missing parse_merge.h?

2018-04-03 18:11 GMT+09:00 Pavan Deolasee : > > > On Tue, Apr 3, 2018 at 2:37 PM, Satoshi Nagayasu wrote: >> >> Hi, >> >> I just got a compile error as below on the latest HEAD. >> --- >> gcc -Wall -Wmissing-prototypes -Wpointer-arith

Re: Missing parse_merge.h?

On Tue, Apr 3, 2018 at 2:37 PM, Satoshi Nagayasu wrote: > Hi, > > I just got a compile error as below on the latest HEAD. > --- > gcc -Wall -Wmissing-prototypes -Wpointer-arith > -Wdeclaration-after-statement -Wendif-labels > -Wmissing-format-attribute

Re: [HACKERS] [PATCH] Incremental sort

On Sun, Apr 1, 2018 at 12:06 AM, Tomas Vondra wrote: > On 03/31/2018 10:43 PM, Tomas Vondra wrote: > > ... > > But I'm pretty sure it may lead to surprising behavior - for example if > > you disable incremental sorts (enable_incrementalsort=off), the plan > > will

Missing parse_merge.h?

Hi, I just got a compile error as below on the latest HEAD. --- gcc -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -fexcess-precision=standard -g -O2 -I. -I.

Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

On 27/03/18 21:02, Tom Lane wrote: Robert Haas writes: On Tue, Mar 27, 2018 at 11:41 AM, Tom Lane wrote: This is ignoring the possibility of damaged data in between, ie A ... B ... CHKPT ... C ... a few zeroed pages ... D ... CHKPT ... E ... F

Re: ALTER TABLE ADD COLUMN fast default

On Fri, Mar 30, 2018 at 10:08 AM, Andrew Dunstan wrote: > On Fri, Mar 30, 2018 at 5:00 AM, Andres Freund wrote: [ missing values are loaded in the TupleDesc by RelationBuildTupleDesc ] >>> > I'm still strongly object to do doing this

  1   2   >