Re: [HACKERS] GUC for cleanup indexes threshold.

On Thu, Apr 5, 2018 at 1:30 AM, Teodor Sigaev wrote: > Thanks for everyone, pushed with minor editorization > Thank you for committing! I found a typo in nbtpage.c and attached a patch fixes it. s/overritten/overwritten/ Regards, -- Masahiko Sawada NIPPON TELEGRAPH AND

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

On Wednesday, April 4, 2018, Amit Kapila wrote: > On Thu, Apr 5, 2018 at 7:14 AM, Andres Freund wrote: > > > > > Questions: > > > > - I'm not perfectly happy with > > "tuple to be locked was already moved to another partition due to > concurrent

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

On Thu, Apr 5, 2018 at 7:14 AM, Andres Freund wrote: > > I've attached a noticeably editorialized patch: > > - I'm uncomfortable with the "moved" information not being crash-safe / > replicated. Thus I added a new flag to preserve it, and removed the > masking of the moved

Re: WIP: a way forward on bootstrap data

Here are the results of an evening's desultory hacking on v13. I was dissatisfied with the fact that we still had several function-referencing columns that had numeric instead of symbolic contents, for instance pg_aggregate.aggfnoid. Of course, the main reason is that those are declared regproc

Unstable number of workers in select_parallel test on spurfowl

Hi, https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=spurfowl=2018-04-05%2003%3A22%3A05 *** /home/pgbuild/buildfarm/HEAD/pgsql.build/src/test/regress/expected/select_parallel.out 2018-04-04 23:22:07.297108347 -0400 ---

Re: Parallel Aggregates for string_agg and array_agg

Hi Tomas, Thanks for taking another look. On 5 April 2018 at 07:12, Tomas Vondra wrote: > Seems fine to me, although we should handle the anyarray case too, I > guess. That is, get_agg_clause_costs_walker() should do this too: > > /* Same thing for

Re: [HACKERS] Runtime Partition Pruning

Hi David. On 2018/03/31 22:52, David Rowley wrote: > The attached patchset is based on Amit's v45 faster partition pruning [1]. > > I've made a few changes since the v14 version. Since Amit's v45 patch > now creates the partition pruning details in a data structure that can > be copied from the

Re: WIP: Covering + unique indexes.

On Wed, Apr 4, 2018 at 3:09 PM, Alexander Korotkov wrote: > Thank you for review! Revised patchset is attached. Cool. * btree_xlog_split() still has this code: /* * On leaf level, the high key of the left page is equal to the first key * on the right

Re: pgsql: New files for MERGE

On Wed, Apr 04, 2018 at 10:10:46AM -0700, Andres Freund wrote: > This needs at the very least a response to the issues pointed out in the > referenced email that you chose to ignore without any sort of comment. That's definitely not cool. -- Michael signature.asc Description: PGP signature

Re: [HACKERS] Restrict concurrent update/delete with UPDATE of partition key

Hi, On 2018-04-02 11:26:38 -0400, Robert Haas wrote: > On Wed, Mar 28, 2018 at 2:12 PM, Andres Freund wrote: > > How will it break it? They'll see an invalid ctid and conclude that the > > tuple is dead? Without any changes that's already something that can > > happen if a

Re: Rewrite of pg_dump TAP tests

On Wed, Apr 04, 2018 at 10:25:03AM -0400, Stephen Frost wrote: > I've updated those tests as well and rebased the patch (though no real > changes were needed for the rebase). Passes all tests. I'll take > another look through the changes again but plan to push them in a few > hours, later on

Re: PATCH: Configurable file mode mask

Hi Michael, On 4/2/18 2:28 AM, Michael Paquier wrote: > > @@ -285,7 +286,7 @@ dsm_impl_posix(dsm_op op, dsm_handle handle, Size > request_size, >* returning. >*/ > flags = O_RDWR | (op == DSM_OP_CREATE ? O_CREAT | O_EXCL : 0); > - if ((fd = shm_open(name, flags, 0600))

Re: [PATCH] btree_gin, add support for uuid, bool, name, bpchar and anyrange types

On 03/30/2018 10:51 PM, Matheus de Oliveira wrote: > Hi all. > > On Wed, Mar 21, 2018 at 1:47 PM, Tomas Vondra > > wrote: > > > Do you plan to post an updated version of the patch, of what is your > response to the

Re: [HACKERS] [PATCH] Lockable views

>> > > +typedef struct >> > > +{ >> > > +Oid root_reloid; >> > > +LOCKMODE lockmode; >> > > +bool nowait; >> > > +Oid viewowner; >> > > +Oid viewoid; >> > > +} LockViewRecurse_context; >> > >> > Probably wouldn't hurt to pgindent the larger changes in the

Re: [PROPOSAL] timestamp informations to pg_stat_statements

> No, the entries are not removed randomly. We track "usage" for each > entry (essentially +1 for each time the query got executed, with a decay > factor applied on each eviction (and we evict 5% at a time). OK I apologize, I hadn't looked in source code in details, and effectively the "Usage"

Re: WIP: a way forward on bootstrap data

Hi, On 2018-04-04 18:29:31 -0400, Tom Lane wrote: > I'm starting to look through v13 seriously, and one thing struck > me that could use some general discussion: what is our policy > going to be for choosing the default values for catalog columns? > > [...] > > In short, I'm tempted to say that

Re: WIP: a way forward on bootstrap data

I'm starting to look through v13 seriously, and one thing struck me that could use some general discussion: what is our policy going to be for choosing the default values for catalog columns? In particular, I noticed that you have for pg_proc boolproisstrict BKI_DEFAULT(f);

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Thu, Apr 5, 2018 at 9:28 AM, Thomas Munro wrote: > On Thu, Apr 5, 2018 at 2:00 AM, Craig Ringer wrote: >> I've tried xfs, jfs, ext3, ext4, even vfat. All behave the same on EIO. >> Didn't try zfs-on-linux or other platforms yet. > > While

Re: comments around heap_lock_tuple confus{ing,ed} around deleted tuples

Hi, On 2018-04-04 18:34:26 -0300, Alvaro Herrera wrote: > > It's also fairly weird that heap_lock_updated_tuple() returns > > /* nothing to lock */ > > return HeapTupleMayBeUpdated; > > when the tuple has been deleted (and thus > > ItemPointerEquals(>t_self, ctid)). That'll not get

Re: comments around heap_lock_tuple confus{ing,ed} around deleted tuples

Andres Freund wrote: > While looking at resolving [1] I re-read heap_lock_tuple() and > subsidiary routines and got thoroughly confused for a while. > > One reason was that function names and comments talk about updated, when > they also actually deal with deletes. heap_lock_updated_tuple() >

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Thu, Apr 5, 2018 at 2:00 AM, Craig Ringer wrote: > I've tried xfs, jfs, ext3, ext4, even vfat. All behave the same on EIO. > Didn't try zfs-on-linux or other platforms yet. I think ZFS will be an outlier here, at least in a pure write()/fsync() test. (1) It doesn't

comments around heap_lock_tuple confus{ing,ed} around deleted tuples

Hi, While looking at resolving [1] I re-read heap_lock_tuple() and subsidiary routines and got thoroughly confused for a while. One reason was that function names and comments talk about updated, when they also actually deal with deletes. heap_lock_updated_tuple() specifically is called on

Re: Foreign keys and partitioned tables

Robert Haas wrote: > I suspect that this leads to bugs under concurrency, something to do > with crosscheck_snapshot, but I couldn't say exactly what the problem > is off the top of my head. My hope is that partitioning might be > immune on the strength of knowing that any given tuple could

Re: pgsql: New files for MERGE

On 4 April 2018 at 21:14, Andres Freund wrote: >> The normal way is to make review comments that allow change. Your >> request for change of the parser data structures is fine and can be >> done, possibly by Saturday > > I did request changes, and you've so far ignored those

Re: pgsql: New files for MERGE

On Wed, Apr 4, 2018 at 1:07 PM, Simon Riggs wrote: > This version works, with agreed semantics, all fully tested and documented. I agree that it's more or less true that this works, and implements the agreed-upon semantics. I also agree that that's very important. That's

Re: pgsql: New files for MERGE

Hi, On 2018-04-04 21:07:25 +0100, Simon Riggs wrote: > It's also neat and tight. Look how easy it was for Peter to add WITH > semantics on top of it. Err. Several parts of the code definitely do not look "neat and tight". As detailed in my email. Possibly that's necessary, but you've not argued

Re: pgsql: New files for MERGE

On 4 April 2018 at 20:09, Tom Lane wrote: > [ removing -committers from cc ] > > Pavan Deolasee writes: >> On Thu, Apr 5, 2018 at 12:16 AM, Andres Freund wrote: >>> Hows that an explanation for just going ahead and committing?

Re: Foreign keys and partitioned tables

Tom Lane wrote: > Alvaro Herrera writes: > > Thanks, pushed. > > This has broken the selinux regression tests, evidently because it > removed ONLY from the emitted FK test queries. While we could change > the expected results, I would first like to hear a defense of why

Re: pgsql: New files for MERGE

On Wed, Apr 4, 2018 at 12:09 PM, Tom Lane wrote: > Personally, I didn't think we had consensus on whether the semantics > are right, let alone on whether this is a satisfactory implementation > code-wise. I know I've never looked at the patch before today; I did not > think

Re: Parallel Aggregates for string_agg and array_agg

On 03/31/2018 04:42 PM, David Rowley wrote: > On 30 March 2018 at 02:55, Tomas Vondra wrote: >> On 03/29/2018 03:09 PM, David Rowley wrote: >>> I meant to mention earlier that I coded >>> agg_args_have_sendreceive_funcs() to only check for send/receive >>>

Re: pgsql: New files for MERGE

[ removing -committers from cc ] Pavan Deolasee writes: > On Thu, Apr 5, 2018 at 12:16 AM, Andres Freund wrote: >> Hows that an explanation for just going ahead and committing? Without >> even commenting on why one thinks the pointed out issues are

Re: Postgres stucks in deadlock detection

Hi, On 2018-04-04 21:55:10 +0300, Teodor Sigaev wrote: > > > > Have you ever observed that in the field? This sounds more artificial > > > > than real to me. > > > > > > Zabbix storage with 50Gb WAL per hour on insertion. > > > > That's not a very detailed description. Isn't that a largely

Re: pgsql: New files for MERGE

On Thu, Apr 5, 2018 at 12:16 AM, Andres Freund wrote: > Hi, > > On 2018-04-05 00:02:06 +0530, Pavan Deolasee wrote: > > Apologies from my end. Simon checked with me regarding your referenced > > email. I was in the middle of responding to it (with a add-on patch to > take > >

Re: Postgres stucks in deadlock detection

Have you ever observed that in the field? This sounds more artificial than real to me. Zabbix storage with 50Gb WAL per hour on insertion. That's not a very detailed description. Isn't that a largely insertion only workload? It's mostly insert load, collecting monitoring information. Also

Re: pgsql: New files for MERGE

On Wed, Apr 4, 2018 at 11:46 AM, Andres Freund wrote: > Hows that an explanation for just going ahead and committing? Without > even commenting on why one thinks the pointed out issues are something > that can be resolved later or somesuch? This has an incredibly rushed >

Re: pgsql: New files for MERGE

Hi, On 2018-04-05 00:02:06 +0530, Pavan Deolasee wrote: > Apologies from my end. Simon checked with me regarding your referenced > email. I was in the middle of responding to it (with a add-on patch to take > care of your review comments), but got side tracked by some high priority > customer

Re: Postgres stucks in deadlock detection

On 2018-04-04 21:34:27 +0300, Teodor Sigaev wrote: > > Have you ever observed that in the field? This sounds more artificial > > than real to me. > > Zabbix storage with 50Gb WAL per hour on insertion. That's not a very detailed description. Isn't that a largely insertion only workload?

Re: Postgres stucks in deadlock detection

Have you ever observed that in the field? This sounds more artificial than real to me. Zabbix storage with 50Gb WAL per hour on insertion. -- Teodor Sigaev E-mail: teo...@sigaev.ru WWW: http://www.sigaev.ru/

Re: pgsql: New files for MERGE

On Wed, Apr 4, 2018 at 10:40 PM, Andres Freund wrote: > Hi, > > On 2018-04-03 08:32:45 -0700, Andres Freund wrote: > > Hi, > > > > On 2018-04-03 09:24:12 +, Simon Riggs wrote: > > > New files for MERGE > > > src/backend/executor/nodeMerge.c | 575

Re: pgsql: Validate page level checksums in base backups

Hi, On Wed, Apr 04, 2018 at 07:25:11PM +0200, Magnus Hagander wrote: > On Wed, Apr 4, 2018 at 3:47 PM, Alvaro Herrera > wrote: > > Michael Banck wrote: > > > > > However, the pg_basebackup testsuite takes up 800+ MB to run, > > > > Uh, you're right. This seems a bit

Re: Postgres stucks in deadlock detection

Hi, On 2018-04-04 11:54:14 +0300, Konstantin Knizhnik wrote: > Several times we and our customers are suffered from the problem that > Postgres got stuck in deadlock detection. > One scenario is YCSB workload with Zipf's distribution when many clients are > trying to update the same record and

Re: Add support for printing/reading MergeAction nodes

On 4 April 2018 at 18:51, Tom Lane wrote: > Simon Riggs writes: >> On 4 April 2018 at 17:19, Tom Lane wrote: >>> If the MERGE patch has broken this, I'm going to push back on that >>> and push back on it hard, because it probably

Re: some last patches breaks plan cache

On 04/04/2018 07:54 PM, Tom Lane wrote: > Tomas Vondra writes: >> This should do the trick - I've failed to realize exec_stmt_call may >> exit by calling elog(ERROR) too, in which case the plan pointer was not >> reset. > >> This does fix the failures presented

Re: some last patches breaks plan cache

Tomas Vondra writes: > This should do the trick - I've failed to realize exec_stmt_call may > exit by calling elog(ERROR) too, in which case the plan pointer was not > reset. > This does fix the failures presented here, but I don't think it's the > right solution

Re: SET TRANSACTION in PL/pgSQL

On 03/29/2018 06:30 PM, Peter Eisentraut wrote: > On 3/15/18 17:49, Alexander Korotkov wrote: >> I didn't dig deeply into this subject.  But should we rather teach SPI >> to execute >> utility statements without taking snapshot when not necessary.  That seems >> like what executor do for client

Re: Add support for printing/reading MergeAction nodes

Simon Riggs writes: > On 4 April 2018 at 17:19, Tom Lane wrote: >> If the MERGE patch has broken this, I'm going to push back on that >> and push back on it hard, because it probably means there are a >> whole bunch of other raw-grammar-output-only node

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Wed, Apr 4, 2018 at 11:23:51PM +0800, Craig Ringer wrote: > On 4 April 2018 at 21:49, Bruce Momjian wrote: > I can understand why kernel developers don't want to keep failed sync > buffers in memory, and once they are gone we lose reporting of their > failure. 

Re: some last patches breaks plan cache

On 04/01/2018 10:01 AM, Pavel Stehule wrote: > > > 2018-04-01 1:00 GMT+02:00 Tomas Vondra >: > > > > On 03/31/2018 08:28 PM, Tomas Vondra wrote: > > > > > > On 03/31/2018 07:56 PM, Tomas Vondra wrote: >

Re: [HACKERS] Runtime Partition Pruning

Hi David, On 04/03/2018 10:10 PM, David Rowley wrote: The attached case doesn't trigger a generic plan, so basically all time is spent in GetCachedPlan. Yeah, there's still no resolution to the fact that a generic plan + runtime pruning might be cheaper than a custom plan. The problem is the

Re: pgsql: Validate page level checksums in base backups

On Wed, Apr 4, 2018 at 3:47 PM, Alvaro Herrera wrote: > Michael Banck wrote: > > > However, the pg_basebackup testsuite takes up 800+ MB to run, > > Uh, you're right. This seems a bit over the top. Can we reduce that > without losing coverage? We've gone great lengths

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 04. 04. 2018 15:49, Bruce Momjian wrote: > I can understand why kernel developers don't want to keep failed sync > buffers in memory, and once they are gone we lose reporting of their > failure. Also, if the kernel is going to not retry the syncs, how long > should it keep reporting the sync

Re: Foreign keys and partitioned tables

Peter Eisentraut wrote: > On 4/3/18 15:11, Alvaro Herrera wrote: > > 0003 is the main patch, which is a bit changed from v4, notably to cover > > your review comments: > > Looks good now. Thanks, pushed. I added a couple of test cases for ON UPDATE/DELETE and MATCH PARTIAL, after noticing that

Re: pgsql: New files for MERGE

Hi, On 2018-04-03 08:32:45 -0700, Andres Freund wrote: > Hi, > > On 2018-04-03 09:24:12 +, Simon Riggs wrote: > > New files for MERGE > > src/backend/executor/nodeMerge.c | 575 +++ > > src/backend/parser/parse_merge.c | 660 > >

Re: Add support for printing/reading MergeAction nodes

Simon Riggs writes: > On 4 April 2018 at 17:19, Tom Lane wrote: >> BTW, poking around in the grammar, I notice that MergeStmt did not >> get added to RuleActionStmt. That seems like a rather serious >> omission. > MERGE isn't a privilege, a trigger

Re: Add support for printing/reading MergeAction nodes

On 4 April 2018 at 17:19, Tom Lane wrote: > BTW, poking around in the grammar, I notice that MergeStmt did not > get added to RuleActionStmt. That seems like a rather serious > omission. MERGE isn't a privilege, a trigger action or a policy action. Why would it be in

Re: Foreign keys and partitioned tables

Robert Haas wrote: > On Sun, Dec 31, 2017 at 2:43 PM, Alvaro Herrera > wrote: > > This patch removes all the ONLY markers from queries in ri_triggers.c. > > That makes the queries work for the new use case, but I haven't figured > > if it breaks things for other use

Re: [HACKERS] GUC for cleanup indexes threshold.

Thanks for everyone, pushed with minor editorization Alexander Korotkov wrote: On Tue, Apr 3, 2018 at 6:42 PM, Masahiko Sawada > wrote: On Tue, Mar 27, 2018 at 8:01 PM, Alexander Korotkov > So, I would like to clarify why could my

Re: [HACKERS] logical decoding of two-phase transactions

Hi, I think the patch looks mostly fine. I'm about to do a bit more testing on it, but a few comments. Attached diff shows which the discussed places / comments more closely. 1) There's a race condition in LogicalLockTransaction. The code does roughly this: if (!BecomeDecodeGroupMember(...))

Re: Add support for printing/reading MergeAction nodes

Marina Polyakova writes: > When debugging is enabled for server logging, isolation tests fail > because there're no corresponding output functions for InsertStmt / > DeleteStmt / UpdateStmt that are used in the output of the MergeAction > nodes (see the attached

Re: [HACKERS] path toward faster partition pruning

Hi, On 04/04/2018 09:29 AM, David Rowley wrote: Thanks for updating. I've made a pass over v49 and I didn't find very much wrong with it. The only real bug I found was a missing IsA(rinfo->clause, Const) in the pseudoconstant check inside generate_partition_pruning_steps_internal. Most of the

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 4 April 2018 at 21:49, Bruce Momjian wrote: > On Wed, Apr 4, 2018 at 07:32:04PM +1200, Thomas Munro wrote: > > On Wed, Apr 4, 2018 at 6:00 PM, Craig Ringer > wrote: > > > On 4 April 2018 at 13:29, Thomas Munro > > >

Add support for printing/reading MergeAction nodes

Hello, hackers! When debugging is enabled for server logging, isolation tests fail because there're no corresponding output functions for InsertStmt / DeleteStmt / UpdateStmt that are used in the output of the MergeAction nodes (see the attached regressions diffs and output). I also attached

Re: Postgres stucks in deadlock detection

So, I see three possible ways to fix this problem: 1. Yury Sololov's patch with two phase deadlock check I could be wrong, but LWLocks in pgsql aren't a "fair" locks: if LWLock is share locked then exclusive lock will wait, but next shared lock will be acquired immediately. If so, the we have a

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 4 April 2018 at 22:25, Bruce Momjian wrote: > On Wed, Apr 4, 2018 at 10:09:09PM +0800, Craig Ringer wrote: > > On 4 April 2018 at 22:00, Craig Ringer wrote: > > > > > > It's the error reporting issues around closing and reopening files > with > >

Re: new function for tsquery creartion

I'm not sure about the different result for these queries: SELECT websearch_to_tsquery('simple', 'cat or '); websearch_to_tsquery -- 'cat' (1 row) SELECT websearch_to_tsquery('simple', 'cat or'); websearch_to_tsquery -- 'cat' & 'or' (1 row) I guess

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Wed, Apr 4, 2018 at 10:09:09PM +0800, Craig Ringer wrote: > On 4 April 2018 at 22:00, Craig Ringer wrote: >   > > It's the error reporting issues around closing and reopening files with > outstanding buffered I/O that's really going to hurt us here. I'll be >

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 4 April 2018 at 22:00, Craig Ringer wrote: > It's the error reporting issues around closing and reopening files with > outstanding buffered I/O that's really going to hurt us here. I'll be > expanding my test case to cover that shortly. > > Also, just to be clear, this

Re: json(b)_to_tsvector with numeric values

Hm, seems, it's useful feature, but I suggest to make separate function jsonb_any_to_tsvector and add support for boolean too (if you know better name for function, do not hide it). Changing behavior of existing function is not obvious for users and, seems, should not backpatched. What do you

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 4 April 2018 at 15:51, Craig Ringer wrote: > On 4 April 2018 at 14:00, Craig Ringer wrote: > >> On 4 April 2018 at 13:29, Thomas Munro >> wrote: >> >>> On Wed, Apr 4, 2018 at 2:44 PM, Thomas Munro >>>

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Wed, Apr 4, 2018 at 10:40:16AM +0800, Craig Ringer wrote: > The trouble with direct I/O is that it pushes a _lot_ of work back on > PostgreSQL regarding knowledge of the storage subsystem, I/O scheduling, etc. > It's absurd to have the kernel do this, unless you want it reliable, in which >

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Wed, Apr 4, 2018 at 07:32:04PM +1200, Thomas Munro wrote: > On Wed, Apr 4, 2018 at 6:00 PM, Craig Ringer wrote: > > On 4 April 2018 at 13:29, Thomas Munro > > wrote: > >> /* Ensure that we skip any errors that predate opening of the file

Re: pgsql: Validate page level checksums in base backups

Michael Banck wrote: > However, the pg_basebackup testsuite takes up 800+ MB to run, Uh, you're right. This seems a bit over the top. Can we reduce that without losing coverage? We've gone great lengths to avoid large amounts of data in tests elsewhere. -- Álvaro Herrera

Re: Foreign keys and partitioned tables

On 4/3/18 15:11, Alvaro Herrera wrote: > 0003 is the main patch, which is a bit changed from v4, notably to cover > your review comments: Looks good now. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: [HACKERS] path toward faster partition pruning

On 4 April 2018 at 19:04, Amit Langote wrote: > On 2018/04/04 14:42, Amit Langote wrote: >> Attached v48. > > I had forgotten to remove the static_pruning parameter I had added in the > v47, because it is no longer used. Static pruning now occurs even if a > step

Re: [HACKERS] MERGE SQL Statement for PG11

Hi Simon, On 03/30/2018 07:10 AM, Simon Riggs wrote: No problems found, but moving proposed commit to 2 April pm There is a warning for this, as attached. Best regards, Jesper diff --git a/src/backend/executor/nodeMerge.c b/src/backend/executor/nodeMerge.c index 0e0d0795d4..9aee073a94

Re: Online enabling of checksums

Hi, On Tue, Apr 03, 2018 at 02:05:04PM +0200, Magnus Hagander wrote: > PFA a rebase on top of the just committed verify-checksums patch. For the record, I am on vacation this week and won't be able to do further in-depth review or testing of this patch before the end of the commitfest, sorry.

Re: pgsql: Validate page level checksums in base backups

Hi, On Wed, Apr 04, 2018 at 11:38:35AM +0200, Magnus Hagander wrote: > On Tue, Apr 3, 2018 at 10:48 PM, Michael Banck > wrote: > > > Hi, > > > > On Tue, Apr 03, 2018 at 08:48:08PM +0200, Magnus Hagander wrote: > > > On Tue, Apr 3, 2018 at 8:29 PM, Tom Lane

Re: Add default role 'pg_access_server_files'

Michael, * Michael Paquier (mich...@paquier.xyz) wrote: > On Mon, Apr 02, 2018 at 05:09:21PM -0400, Stephen Frost wrote: > > * Michael Paquier (mich...@paquier.xyz) wrote: > >> No refactoring for pg_file_unlink and its v1.1? > > > > I considered each function and thought about if it'd make sense

Re: Optimize Arm64 crc32c implementation in Postgresql

On Wed, Apr 4, 2018 at 11:47 PM, Thomas Munro wrote: > BTW I did some googling just now and found out that some libraries use > a technique they call "CPU probing": just try it and see if you get > SIGILL. Is that a bad idea for some reason? Here is a quick hack

Re: Optimize Arm64 crc32c implementation in Postgresql

On Wed, Apr 4, 2018 at 11:21 PM, Heikki Linnakangas wrote: > Yep. I got the code before the main loop, to handle the first 1-7 unaligned > bytes, wrong. Apparently those are the only tests that call the CRC function > with very short and unaligned input. BTW I did some googling

Re: Optimize Arm64 crc32c implementation in Postgresql

On 04/04/18 14:13, Thomas Munro wrote: On Wed, Apr 4, 2018 at 9:23 PM, Heikki Linnakangas wrote: Pushed, thanks everyone! On eelpout two test_decoding tests failed: test ddl ... FAILED test rewrite ... FAILED The output has

Re: Optimize Arm64 crc32c implementation in Postgresql

On Wed, Apr 4, 2018 at 9:23 PM, Heikki Linnakangas wrote: > Pushed, thanks everyone! On eelpout two test_decoding tests failed: test ddl ... FAILED test rewrite ... FAILED The output has several cases where

Re: [HACKERS] Add support for tuple routing to foreign partitions

(2018/04/03 22:01), Etsuro Fujita wrote: Attached is an updated version of the patch. Patch foreign-routing-fdwapi-3.patch is created on top of patch postgres-fdw-refactoring-3.patch and the bug-fix patch [1]. One thing I noticed about patch foreign-routing-fdwapi-3.patch is this bug: the

Re: [HACKERS] GUC for cleanup indexes threshold.

On Tue, Apr 3, 2018 at 6:42 PM, Masahiko Sawada wrote: > On Tue, Mar 27, 2018 at 8:01 PM, Alexander Korotkov > > So, I would like to clarify why could my patch block future improvements > > in this area? For instance, if we would decide to make btree able to > skip > >

Re: pgsql: Validate page level checksums in base backups

On Tue, Apr 3, 2018 at 10:48 PM, Michael Banck wrote: > Hi, > > On Tue, Apr 03, 2018 at 08:48:08PM +0200, Magnus Hagander wrote: > > On Tue, Apr 3, 2018 at 8:29 PM, Tom Lane wrote: > > I'd bet a good lunch that nondefault BLCKSZ would break it, as

Re: Optimize Arm64 crc32c implementation in Postgresql

On 03/04/18 19:43, Andres Freund wrote: Architecture manual time? They're available freely IIRC and should answer this. Yeah. The best reference I could find was "ARM Cortex-A Series Programmer’s Guide for ARMv8-A" (http://infocenter.arm.com/help/topic/com.arm.doc.den0024a/ch08s01.html).

Postgres stucks in deadlock detection

Hi hackers, Please notice that it is not a first of April joke;) Several times we and our customers are suffered from the problem that Postgres got stuck in deadlock detection. One scenario is YCSB workload with Zipf's distribution when many clients are trying to update the same record and

Re: Optimize Arm64 crc32c implementation in Postgresql

On 03/04/18 21:56, Daniel Gustafsson wrote: The following line should say SSE and not SSSE (and the same typo is in src/include/pg_config.h.in and src/include/pg_config.h.win32). While not introduced in this patch, it’s adjacent to the patched codepath and on topic so may well be fixed while in

Re: Problem while setting the fpw with SIGHUP

At Sat, 31 Mar 2018 17:43:58 +0530, Amit Kapila wrote in > On Wed, Mar 28, 2018 at 12:10 PM, Kyotaro HORIGUCHI > wrote: > > At Tue, 27 Mar 2018 22:02:26 +0900, Michael

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 4 April 2018 at 14:00, Craig Ringer wrote: > On 4 April 2018 at 13:29, Thomas Munro > wrote: > >> On Wed, Apr 4, 2018 at 2:44 PM, Thomas Munro >> wrote: >> > On Wed, Apr 4, 2018 at 2:14 PM, Bruce Momjian

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On Wed, Apr 4, 2018 at 6:00 PM, Craig Ringer wrote: > On 4 April 2018 at 13:29, Thomas Munro > wrote: >> /* Ensure that we skip any errors that predate opening of the file */ >> f->f_wb_err = filemap_sample_wb_err(f->f_mapping); >> >> [...] >

Re: [HACKERS] Runtime Partition Pruning

On 2018/04/04 16:04, David Rowley wrote: > On 4 April 2018 at 18:27, Amit Langote wrote: >> I'm not sure if we've yet discussed anything that'd be related to this on >> the faster pruning thread. > > hmm, yeah, I didn't really explain the context, but the report

Re: [HACKERS] path toward faster partition pruning

On 4 April 2018 at 19:04, Amit Langote wrote: > Attached v49. Thank for including the changes. I'll look now. -- David Rowley http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services

Re: [HACKERS] path toward faster partition pruning

On 4 April 2018 at 17:42, Amit Langote wrote: > I'm not sure about the following change in your patch: > > - if (!result->scan_null) > - result->scan_null = step_result->scan_null; > - if (!result->scan_default) > -

Re: [HACKERS] Runtime Partition Pruning

On 4 April 2018 at 18:27, Amit Langote wrote: > On 2018/04/04 11:10, David Rowley wrote: >> On 4 April 2018 at 05:44, Jesper Pedersen wrote: >>> Also, I'm seeing a regression for check-world in >>> src/test/regress/results/inherit.out

Re: [PATCH] Logical decoding of TRUNCATE

On 4 April 2018 at 03:31, Peter Eisentraut wrote: > I wonder why this approach was chosen. I looked at coding it that way and it seemed worse than the way chosen. > I'm going to try to hack up an alternative approach and see how it works > out. If you add a

Re: [HACKERS] Runtime Partition Pruning

Hi David. On 2018/04/04 11:10, David Rowley wrote: > On 4 April 2018 at 05:44, Jesper Pedersen wrote: >> Also, I'm seeing a regression for check-world in >> src/test/regress/results/inherit.out >> >> *** >> *** 642,648 >>

Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

On 4 April 2018 at 13:29, Thomas Munro wrote: > On Wed, Apr 4, 2018 at 2:44 PM, Thomas Munro > wrote: > > On Wed, Apr 4, 2018 at 2:14 PM, Bruce Momjian wrote: > >> Uh, are you sure it fixes our use-case? From the