Re: [HACKERS] [COMMITTERS] pgsql: Fix cardinality estimates for parallel joins.

2017-03-15 Thread Amit Kapila
On Tue, Mar 14, 2017 at 9:59 PM, Robert Haas wrote: > On Tue, Jan 17, 2017 at 11:49 AM, Robert Haas wrote: >> On Mon, Jan 16, 2017 at 7:23 AM, Amit Kapila wrote: >>> >>> >>> Isn't it better to call clamp_row_est in join costing functions as we >>> are doing in cost_seqscan()? Is there a reason

Re: [HACKERS] PATCH: Configurable file mode mask

2017-03-15 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of David Steele > > But it might be worth thinking about whether we want to encourage > > people to do manual chmod's at all; that's fairly easy to get wrong, > > particularly given the difference in X

Re: [HACKERS] Defaulting psql to ON_ERROR_ROLLBACK=interactive

2017-03-15 Thread Tsunakawa, Takayuki
From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Peter van > Hardenberg > I suggest we update the default of ON_ERROR_ROLLBACK to interactive for > 10.0. +1 Regards Takayuki Tsunakawa -- Sent via pgsql-hackers mailing list (pgsql-hackers@postg

Re: [HACKERS] Measuring replay lag

2017-03-15 Thread Ian Barwick
Hi Just adding a couple of thoughts on this. On 03/14/2017 08:39 AM, Thomas Munro wrote: > Hi, > > Please see separate replies to Simon and Craig below. > > On Sun, Mar 5, 2017 at 8:38 PM, Simon Riggs wrote: >> On 1 March 2017 at 10:47, Thomas Munro wrote: >>> I do see why a new user trying th

Re: [HACKERS] New procedural language

2017-03-15 Thread Massimo Fidanza
But there is a difference between python and Go or Rust that the first is interpreted and the seconds are compiled like C. All PL instead of C and Java (also byte code interpreted) are interpreted Il 15 mar 2017 3:52 AM, "Amit Langote" ha scritto: > Hi, > > On 2017/03/15 11:08, Massimo Fidanza w

Re: [HACKERS] wait events for disk I/O

2017-03-15 Thread Rushabh Lathia
Thanks Rahila for reviewing this patch. On Tue, Mar 14, 2017 at 8:13 PM, Rahila Syed wrote: > Hello, > > I applied and tested this patch on latest sources and it works fine. > > Following are some comments, > > >+ /* Wait event for SNRU */ > >+ WAIT_EVENT_READ_SLRU_PAGE, > Typo in the commen

Re: [HACKERS] logical decoding of two-phase transactions

2017-03-15 Thread Petr Jelinek
On 02/03/17 17:34, Petr Jelinek wrote: > On 02/03/17 13:23, Craig Ringer wrote: >> On 2 March 2017 at 16:20, Stas Kelvich wrote: >>> On 2 Mar 2017, at 11:00, Craig Ringer wrote: We already have it, because we just decoded the PREPARE TRANSACTION. I'm preparing a patch revisi

Re: [HACKERS] New procedural language

2017-03-15 Thread Amit Langote
Hi, On 2017/03/15 16:20, Massimo Fidanza wrote: > Il 15 mar 2017 3:52 AM, "Amit Langote" ha > scritto: >> On 2017/03/15 11:08, Massimo Fidanza wrote: >>> Hello, >>> >>> what it means to add a new procedural language such as Go or Rust? >> >> I think you're looking for how to write a "PL function

Re: [HACKERS] Speedup twophase transactions

2017-03-15 Thread Nikhil Sontakke
> Thanks for the new version. Let's head toward a final patch. > > :-) > > > > Have added a new function to do this now. It reads either from disk or > > shared memory and produces error/log messages accordingly as well now. > > And that's ProcessTwoPhaseBufferAndReturn in the patch. > ProcessTw

Re: [HACKERS] [PATCH] Suppress Clang 3.9 warnings

2017-03-15 Thread Aleksander Alekseev
Hi Hoah. Thanks a lot for a reply! > This is wrong on platforms that do have strlcpy() in libc. If it no too much trouble could you please explain what will happen on such platforms? On what platform did you check it? I'm sure it fixable. And I got a strong feeling that "wrong" could be a bit ex

Re: [HACKERS] New CORRESPONDING clause design

2017-03-15 Thread Pavel Stehule
Hi 2017-03-14 16:33 GMT+01:00 Surafel Temesgen : > > hi > > Some errors are related to just CORRESPONDING without any columns. So >> using expr doesn't help here. So parse node CORRESPONDING can solve both >> issues. >> > In current implementation pointing to a node means pointing to a node’s >

Re: [HACKERS] Report the number of skipped frozen pages by manual VACUUM

2017-03-15 Thread Yugo Nagata
On Fri, 10 Mar 2017 20:08:42 +0900 Masahiko Sawada wrote: > On Fri, Mar 10, 2017 at 3:58 PM, Jim Nasby wrote: > > On 3/6/17 8:34 PM, Masahiko Sawada wrote: > >> > >> I don't think it can say "1 frozen pages" because the number of > >> skipped pages according to visibility map is always more than

Re: [HACKERS] background sessions

2017-03-15 Thread Pavel Stehule
2017-03-15 0:44 GMT+01:00 Robert Haas : > On Tue, Mar 14, 2017 at 4:54 PM, Pavel Stehule > wrote: > >> I don't understand. The only way you'd need a server restart is if a > >> background process wasn't responding to SIGTERM, and that's a bug > >> independent of anything this patch does. It wou

Re: [HACKERS] Backend crash on non-exclusive backup cancel

2017-03-15 Thread Anastasia Lubennikova
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: tested, passed Documentation:tested, passed As I see, this bugfix works as expected and the patch is smal

Re: [HACKERS] Report the number of skipped frozen pages by manual VACUUM

2017-03-15 Thread Masahiko Sawada
On Wed, Mar 15, 2017 at 1:09 PM, Yugo Nagata wrote: > On Fri, 10 Mar 2017 20:08:42 +0900 > Masahiko Sawada wrote: > >> On Fri, Mar 10, 2017 at 3:58 PM, Jim Nasby wrote: >> > On 3/6/17 8:34 PM, Masahiko Sawada wrote: >> >> >> >> I don't think it can say "1 frozen pages" because the number of >> >

Re: [HACKERS] Remove obsolete text from hash/README

2017-03-15 Thread Robert Haas
On Tue, Mar 14, 2017 at 10:22 PM, Amit Kapila wrote: > As pointed out by Tom [1], attached is a patch to remove obsolete text > from src/backend/access/hash/README Heh, thanks. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hacker

Re: [HACKERS] GSOC - TOAST'ing in slices

2017-03-15 Thread Robert Haas
On Tue, Mar 14, 2017 at 10:03 PM, George Papadrosou wrote: > The project’s idea is implement different slicing approaches according to > the value’s datatype. For example a text field could be split upon character > boundaries while a JSON document would be split in a way that allows fast > access

Re: [HACKERS] IF NOT EXISTS option for CREATE SERVER and CREATE USER MAPPING statements

2017-03-15 Thread Arthur Zakirov
2017-03-14 15:55 GMT+03:00 Ashutosh Bapat : > > I noticed that the earlier error message was using "server" instead of > "foreign server", while the new message uses the later one. Usually, > when converting an error to notice, we don't expect such changes. But > many other error messages are using

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-03-15 Thread Sachin Kotwal
Thanks for nice patch related to AWS RDS. Can we have backpatch this patch to PostgreSQL 9.6 and earlier releases ? Regards, Sachin On Sun, Mar 5, 2017 at 12:30 PM, Simon Riggs wrote: > On 28 February 2017 at 17:49, Simon Riggs wrote: > > > I've edited the stated reason for the patch on the

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-15 Thread Kuntal Ghosh
On Tue, Mar 14, 2017 at 1:50 PM, Michael Paquier wrote: > "writer" would be better if defined as "background writer" instead? > You are forgetting in this list autovacuum workers and the startup > process, the latter is important for nodes in recovery. > Modified "writer" as "background writer". I

Re: [HACKERS] GSOC - TOAST'ing in slices

2017-03-15 Thread Alexander Korotkov
On Wed, Mar 15, 2017 at 5:03 AM, George Papadrosou wrote: > *Deliverables * > > - Implement “semantic” slicing for datatypes that support slicing into > TOAST tables. These datatypes will be the Text, Array, JSON/JSONb and XML > data types. > That looks too much comprehensive GSoC project for m

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Amit Kapila
On Wed, Mar 15, 2017 at 1:15 AM, Robert Haas wrote: > On Tue, Mar 14, 2017 at 8:02 AM, Ashutosh Sharma > wrote: >> Attached is the v6 patch for microvacuum in hash index rebased on top >> of 'v10 patch for WAL in hash index - [1]' and 'v1 patch for WAL >> consistency check for hash index - [2]'.

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-15 Thread Ashutosh Bapat
>> >> There are some differences in what geqo does and what partition-wise >> needs to do. geqo tries many joining orders each one in a separate >> temporary context. The way geqo slices the work, every slice produces >> a full plan. For partition-wise join I do not see a way to slice the >> work s

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Stephen Frost
Amit, * Amit Kapila (amit.kapil...@gmail.com) wrote: > On Wed, Mar 15, 2017 at 12:53 AM, Stephen Frost wrote: > > If that's the case then > > this does seem to at least be less of an issue, though I hope we put in > > appropriate comments about it. > > I think we have sufficient comments in cod

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-03-15 Thread Stephen Frost
Greetings, * Sachin Kotwal (kotsac...@gmail.com) wrote: > Can we have backpatch this patch to PostgreSQL 9.6 and earlier releases ? No. This is a new feature and new features are not back-patched. Thanks! Stephen signature.asc Description: Digital signature

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-03-15 Thread Amit Kapila
On Thu, Mar 9, 2017 at 10:21 PM, Masahiko Sawada wrote: > On Wed, Mar 8, 2017 at 1:43 AM, Peter Geoghegan wrote: >> On Sat, Mar 4, 2017 at 1:30 AM, Amit Kapila wrote: While I can't see this explained anywhere, I'm pretty sure that that's supposed to be impossible, which this patch

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-03-15 Thread Sachin Kotwal
Hi Stephen, Thanks. I understand this is small but new feature and not bug fix. But we should be able to backpatch if there is no dependency. It will help users to get benefit of this feature for g96 and pg95 in RDS until they will have pg10 in RDS. If It is against community policy then it is

Re: [HACKERS] Allow pg_dumpall to work without pg_authid

2017-03-15 Thread Stephen Frost
Greetings, * Sachin Kotwal (kotsac...@gmail.com) wrote: > Thanks. I understand this is small but new feature and not bug fix. > But we should be able to backpatch if there is no dependency. No, it's a new feature and won't be back-patched. > It will help users to get benefit of this feature for

Re: [HACKERS] GSOC - TOAST'ing in slices

2017-03-15 Thread Tom Lane
Robert Haas writes: > On Tue, Mar 14, 2017 at 10:03 PM, George Papadrosou > wrote: >> The project’s idea is implement different slicing approaches according to >> the value’s datatype. For example a text field could be split upon character >> boundaries while a JSON document would be split in a w

Re: [HACKERS] Defaulting psql to ON_ERROR_ROLLBACK=interactive

2017-03-15 Thread Tom Lane
Peter van Hardenberg writes: > It's quite clear to me that the current default behaviour costs the world > serious pain and heart-ache on a daily basis. This is a large claim for which you've provided little evidence. On the other hand, changing behavior like this from one version to the next wi

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 9:18 AM, Stephen Frost wrote: >> I think we have sufficient comments in code especially on top of >> function _hash_alloc_buckets(). > > I don't see any comments regarding how we have to be sure to handle > an out-of-space case properly in the middle of a file because we've

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Tom Lane
Robert Haas writes: > Now, that having been said, I'm not sure it's a good idea to tinker > with the behavior for v10. We could change the new-splitpoint code so > that it loops over all the pages in the new splitpoint and zeroes them > all, instead of just the last one. Why would we do that? T

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Emre Hasegeli
> I don't know if this is the only problem > I'll be in this general area today, so will mention if I stumble over > anything that looks broken. I was testing the same patch with a large dataset and got a different segfault: > hasegeli=# explain select * from only mp_notification_20170225 where

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Tom Lane
Stephen Frost writes: > I do see that mdwrite() should handle an out-of-disk-space case, though > that just makes me wonder what's different here compared to normal > relations that we don't have an issue with a sparse WAL'd hash index but > we can't handle it if a normal relation is sparse. *Any

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > FWIW, I'm not certain that Stephen is correct to claim that we have > some concrete problem with sparse files. We certainly don't *depend* > on sparse storage anyplace else, nor write data in a way that would be > likely to trigger it; but I'm not awa

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 10:34 AM, Tom Lane wrote: > FWIW, I'm not certain that Stephen is correct to claim that we have > some concrete problem with sparse files. We certainly don't *depend* > on sparse storage anyplace else, nor write data in a way that would be > likely to trigger it; but I'm n

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Robert Haas
On Tue, Mar 14, 2017 at 10:30 PM, Amit Kapila wrote: > On Tue, Mar 14, 2017 at 10:59 PM, Robert Haas wrote: >> On Mon, Mar 13, 2017 at 11:48 PM, Amit Kapila >> wrote: >>> We didn't found any issue with the above testing. >> >> Great! I've committed the latest version of the patch, with some >>

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Stephen Frost
Robert, * Robert Haas (robertmh...@gmail.com) wrote: > On Wed, Mar 15, 2017 at 10:34 AM, Tom Lane wrote: > > FWIW, I'm not certain that Stephen is correct to claim that we have > > some concrete problem with sparse files. We certainly don't *depend* > > on sparse storage anyplace else, nor write

Re: [HACKERS] [PATCH] Suppress Clang 3.9 warnings

2017-03-15 Thread Tom Lane
Noah Misch writes: > On Mon, Mar 13, 2017 at 06:35:53PM +0300, Aleksander Alekseev wrote: >> + * Unfortunately in case of strlcat and strlcpy we can't trust tests >> + * executed by Autotools if Clang > 3.6 is used. > This is wrong on platforms that do have strlcpy() in libc. Didn't you submit a

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Tom Lane
Robert Haas writes: > That theory seems inconsistent with how mdextend() works. My > understanding is that we zero-fill the new blocks before populating > them with actual data precisely to avoid running out of disk space due > to deferred allocation at the OS level. If we don't care about > fai

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-15 Thread Rafia Sabih
On Wed, Mar 15, 2017 at 10:52 AM, Dilip Kumar wrote: > I have reviewed the patch, I have some questions. > > @@ -3031,7 +3031,7 @@ exec_stmt_return_query(PLpgSQL_execstate *estate, > Assert(stmt->dynquery != NULL); > portal = exec_dynquery_with_params(estate, stmt->dynquery, > stmt->params

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Stephen Frost
Tom, * Tom Lane (t...@sss.pgh.pa.us) wrote: > Robert Haas writes: > > That theory seems inconsistent with how mdextend() works. My > > understanding is that we zero-fill the new blocks before populating > > them with actual data precisely to avoid running out of disk space due > > to deferred al

Re: [HACKERS] scram and \password

2017-03-15 Thread Robert Haas
On Tue, Mar 14, 2017 at 5:14 PM, Tom Lane wrote: >> Without md5-only, a user who uses \password to change their password from a >> newer client would lock themselves out of connecting again from older >> clients. As a conscious decision (either of the DBA or the user) that >> would be OK, but to

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 11:02 AM, Tom Lane wrote: > Robert Haas writes: >> That theory seems inconsistent with how mdextend() works. My >> understanding is that we zero-fill the new blocks before populating >> them with actual data precisely to avoid running out of disk space due >> to deferred

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-15 Thread Kevin Grittner
On Tue, Mar 14, 2017 at 3:45 PM, Kevin Grittner wrote: >> On 3/14/17 17:34, Mengxing Liu wrote: >>> There are several alternative benchmarks. Tony suggested that we >>> should use TPC-E and TPC-DS. > > More benchmarks is better, all other things being equal. Keep in > mind that good benchmarking

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Dilip Kumar
On Wed, Mar 15, 2017 at 8:11 PM, Emre Hasegeli wrote: > > I can try to provide a test case, if that wouldn't be enough to spot > the problem. Thanks for reporting, I am looking into this. Meanwhile, if you can provide the reproducible test case then locating the issue will be faster. -- Regard

Re: [HACKERS] Enabling parallelism for queries coming from SQL or other PL functions

2017-03-15 Thread Robert Haas
On Fri, Mar 10, 2017 at 7:08 AM, Rafia Sabih wrote: > I wanted to clarify a few things here, I noticed that call of ExecutorRun in > postquel_getnext() uses !es->lazyEval as execute_once, this is confusing, as > it is true even in cases when a simple query like "select count(*) from t" > is used i

Re: [HACKERS][REVIEW] macaddr 64 bit (EUI-64) datatype support

2017-03-15 Thread Stephen Frost
Greetings Hari Babu, * Haribabu Kommi (kommi.harib...@gmail.com) wrote: > On Mon, Mar 13, 2017 at 6:52 AM, Stephen Frost wrote: > > And, naturally, re-reading the email as it hit the list made me realize > > that the documentation/error-message incorrectly said "3rd and 4th" > > bytes were being

Re: [HACKERS] Defaulting psql to ON_ERROR_ROLLBACK=interactive

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 2:29 AM, Peter van Hardenberg wrote: > Ads and I were talking over breakfast about usability issues and he > mentioned transaction cancellation during interactive sessions as a serious > pain point. > > I suggest we update the default of ON_ERROR_ROLLBACK to interactive for

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
> Generally, this patch looks like a pretty straightforward adaptation > of the similar btree mechanism to hash indexes, so if it works for > btree it ought to work here, too. But I noticed a few things while > reading through it. > > +/* Get RelfileNode from relation OID */ > +rel

Re: [HACKERS] Backend crash on non-exclusive backup cancel

2017-03-15 Thread David Steele
On 3/15/17 2:28 AM, Michael Paquier wrote: > On Wed, Mar 15, 2017 at 12:27 AM, Anastasia Lubennikova > wrote: >> As far as I understand, in this thread were discussed two bugs of >> pg_stop_backup(). >> Thanks to the clear descriptions above, I easily reproduced both of them. >> >> BUG#1: >> Serv

Re: [HACKERS] Defaulting psql to ON_ERROR_ROLLBACK=interactive

2017-03-15 Thread Pavel Stehule
2017-03-15 16:38 GMT+01:00 Robert Haas : > On Wed, Mar 15, 2017 at 2:29 AM, Peter van Hardenberg wrote: > > Ads and I were talking over breakfast about usability issues and he > > mentioned transaction cancellation during interactive sessions as a > serious > > pain point. > > > > I suggest we up

Re: [HACKERS] Backend crash on non-exclusive backup cancel

2017-03-15 Thread Andres Freund
Hi, On 2017-03-15 15:28:03 +0900, Michael Paquier wrote: > diff --git a/src/backend/access/transam/xlog.c > b/src/backend/access/transam/xlog.c > index 64335f909e..eaf8e32fe1 100644 > --- a/src/backend/access/transam/xlog.c > +++ b/src/backend/access/transam/xlog.c > --- a/src/backend/access/tran

Re: [HACKERS] scram and \password

2017-03-15 Thread Michael Paquier
On Wed, Mar 15, 2017 at 6:14 AM, Tom Lane wrote: > Jeff Janes writes: >> On Tue, Mar 14, 2017 at 8:40 AM, Tom Lane wrote: >>> Why exactly would anyone want "md5 only"? I should think that "scram >>> only" is a sensible pg_hba setting, if the DBA feels that md5 is too >>> insecure, but I do not

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
> > I think one possibility is to get it using > indexrel->rd_index->indrelid in _hash_doinsert(). > Thanks. I have tried the same in the v7 patch shared upthread. > >> >> But they're not called delete records in a hash index. The function >> is called hash_xlog_vacuum_one_page. The correspondi

Re: [HACKERS] SQL/JSON in PostgreSQL

2017-03-15 Thread David Steele
On 3/7/17 11:05 PM, David Steele wrote: > On 3/7/17 11:38 AM, Andres Freund wrote: > > <...> > >>> We have a plenty of time and we dedicate one full-time developer for >>> this project. >> >> How about having that, and perhaps others, developer participate in >> reviewing patches and getting to t

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 11:37 AM, Ashutosh Sharma wrote: >> +/* Get RelfileNode from relation OID */ >> +rel = relation_open(htup->t_tableOid, NoLock); >> +rnode = rel->rd_node; >> +relation_close(rel, NoLock); >> itup->t_tid = htup->t_self; >> -_ha

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Dilip Kumar
On Wed, Mar 15, 2017 at 8:51 PM, Dilip Kumar wrote: >> I can try to provide a test case, if that wouldn't be enough to spot >> the problem. > > Thanks for reporting, I am looking into this. Meanwhile, if you can > provide the reproducible test case then locating the issue will be > faster. After

Re: [HACKERS] [PATCH] Suppress Clang 3.9 warnings

2017-03-15 Thread Noah Misch
On Wed, Mar 15, 2017 at 10:57:15AM -0400, Tom Lane wrote: > Noah Misch writes: > > On Mon, Mar 13, 2017 at 06:35:53PM +0300, Aleksander Alekseev wrote: > >> + * Unfortunately in case of strlcat and strlcpy we can't trust tests > >> + * executed by Autotools if Clang > 3.6 is used. > > > This is w

[HACKERS] Re: new set of psql patches for loading (saving) data from (to) text, binary files

2017-03-15 Thread Stephen Frost
Pavel, I started looking through this to see if it might be ready to commit and I don't believe it is. Below are my comments about the first patch, I didn't get to the point of looking at the others yet since this one had issues. * Pavel Stehule (pavel.steh...@gmail.com) wrote: > 2017-01-09 17:2

Re: [HACKERS] [PATCH] Suppress Clang 3.9 warnings

2017-03-15 Thread Tom Lane
Noah Misch writes: > On Wed, Mar 15, 2017 at 10:57:15AM -0400, Tom Lane wrote: >> Seems like the correct solution is to >> absorb that fix, either by updating to a newer autoconf release or by >> carrying our own version of AC_CHECK_DECLS until they come out with one. > As you mention upthread, t

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Dilip Kumar
On Wed, Mar 15, 2017 at 8:11 PM, Emre Hasegeli wrote: >> * thread #1: tid = 0x5045a8f, 0x00010ae44558 >> postgres`brin_deform_tuple(brdesc=0x7fea3c86a3a8, >> tuple=0x7fea3c891040) + 40 at brin_tuple.c:414, queue = >> 'com.apple.main-thread', stop reason = signal SIGUSR1 >> * frame

Re: [HACKERS] [POC] hash partitioning

2017-03-15 Thread Robert Haas
On Tue, Mar 14, 2017 at 10:08 AM, David Steele wrote: > This patch is marked as POC and after a read-through I agree that's > exactly what it is. Just out of curiosity, were you looking at Nagata-san's patch, or Amul's? > As such, I'm not sure it belongs in the last > commitfest. Furthermore, t

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Emre Hasegeli
> This can crash at line:414, if either tuple is invalid memory(but I > think it's not because we have already accessed this memory in above > if check) or dtup is invalid (this is also not possible because > brin_new_memtuple has already accessed this). I was testing with the brin correlation pat

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-15 Thread Mengxing Liu
> -Original Messages- > From: "Kevin Grittner" > Sent Time: 2017-03-15 23:20:07 (Wednesday) > To: DEV_OPS > Cc: "Mengxing Liu" , > "pgsql-hackers@postgresql.org" > Subject: Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from > rw-conflict tracking in serializable transactions

Re: [HACKERS] [POC] hash partitioning

2017-03-15 Thread David Steele
On 3/15/17 12:25 PM, Robert Haas wrote: > On Tue, Mar 14, 2017 at 10:08 AM, David Steele wrote: >> This patch is marked as POC and after a read-through I agree that's >> exactly what it is. > > Just out of curiosity, were you looking at Nagata-san's patch, or Amul's? Both - what I was looking fo

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Dilip Kumar
On Wed, Mar 15, 2017 at 10:02 PM, Emre Hasegeli wrote: > I was testing with the brin correlation patch [1] applied. I cannot > crash it without the patch either. I am sorry for not testing it > before. The patch make BRIN selectivity estimation function access > more information. > > [1] > htt

Re: [HACKERS] [COMMITTERS] pgsql: Use asynchronous connect API in libpqwalreceiver

2017-03-15 Thread Tom Lane
Andrew Dunstan writes: > On 03/03/2017 11:11 PM, Tom Lane wrote: >> Yeah, I was wondering if this is just exposing a pre-existing bug. >> However, the "normal" path operates by repeatedly invoking PQconnectPoll >> (cf. connectDBComplete) so it's not immediately obvious how such a bug >> would've e

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Emre Hasegeli
> With my test case, I could not crash even with this patch applied. > Can you provide your test case? Yes: > hasegeli=# create table r2 as select (random() * 3)::int as i from > generate_series(1, 100); > SELECT 100 > hasegeli=# create index on r2 using brin (i); > CREATE INDEX > hasege

Re: [HACKERS] background sessions

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 6:43 AM, Pavel Stehule wrote: > I don't understand - CHECK_FOR_INTERRUPTS called from executor implicitly. True. So there shouldn't be any problem here. I'm confused as can be about what you want changed. Some review of the patch itself: +pq_redirect_to_shm_mq(sess

Re: [HACKERS] [POC] hash partitioning

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 12:39 PM, David Steele wrote: > Agreed. Perhaps both types of syntax should be supported, one that is > friendly to users and one that is precise for dump tools and those who > care get in the weeds. Eventually, sure. For the first version, I want to skip the friendly sy

Re: [HACKERS] identity columns

2017-03-15 Thread Peter Eisentraut
Vitaly, will you be able to review this again? On 2/28/17 21:23, Peter Eisentraut wrote: > New patch that fixes everything. ;-) Besides hopefully addressing all > your issues, this version also no longer requires separate privileges on > the internal sequence, which was the last outstanding fun

Re: [HACKERS] logical replication access control patches

2017-03-15 Thread Peter Eisentraut
On 3/14/17 15:05, Stephen Frost wrote: > Another approach to solving my concern would be to only allow the > publishing of tables by non-owner users who have table-level SELECT > rights An early version of the logical replication patch set did that. But the problem is that this way someone with S

Re: [HACKERS] logical replication access control patches

2017-03-15 Thread Peter Eisentraut
On 3/14/17 15:37, Petr Jelinek wrote: > Yeah that's rather hard to say in front. Maybe safest action would be to > give the permission to owners in 10 and revisit special privilege in 11 > based on feedback? I'm fine with that. -- Peter Eisentraut http://www.2ndQuadrant.com/ Postgre

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-15 Thread Peter Eisentraut
make check-world -O -j6 PROVE_FLAGS=-j6 2 min 34 seconds Nice! -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subs

Re: [HACKERS] pg_ls_waldir() & pg_ls_logdir()

2017-03-15 Thread Robert Haas
On Mon, Feb 20, 2017 at 6:21 AM, Dave Page wrote: > Patch includes the code and doc updates. Review: +strftime(mtime, 25, "%Y-%m-%d %H:%M:%S %Z", localtime(&(attrib.st_ctime))); +const int n = snprintf(NULL, 0, "%lld", attrib.st_size); +char size[n+1]; +snprintf(s

Re: [HACKERS] logical replication access control patches

2017-03-15 Thread Peter Eisentraut
On 3/14/17 14:49, Petr Jelinek wrote: > Not what I mean - owner should be able to publish table. If you are > granted role of the owner you can do what owner can no? I didn't actually know that ownership worked that way. You can grant the role of an owner to someone, and then that someone has own

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Dilip Kumar
On Wed, Mar 15, 2017 at 10:21 PM, Emre Hasegeli wrote: >> hasegeli=# create table r2 as select (random() * 3)::int as i from >> generate_series(1, 100); >> SELECT 100 >> hasegeli=# create index on r2 using brin (i); >> CREATE INDEX >> hasegeli=# analyze r2; >> ANALYZE >> hasegeli=# explai

Re: [HACKERS] Need a builtin way to run all tests faster manner

2017-03-15 Thread Tom Lane
Peter Eisentraut writes: > make check-world -O -j6 PROVE_FLAGS=-j6 > 2 min 34 seconds > Nice! One thing I've noticed with parallel check-world is that it's possible for a sub-job to fail and for the resulting complaint from make to scroll offscreen before everything stops, leaving all the visible

Re: [HACKERS] [COMMITTERS] pgsql: Improve isolation tests infrastructure.

2017-03-15 Thread Andres Freund
Hi Peter, All, On 2017-03-14 23:10:19 +, Andres Freund wrote: > Improve isolation tests infrastructure. There's: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=frogmouth&dt=2017-03-15%2012%3A32%3A45 https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=narwhal&dt=2017-03-15%2005%3A

Re: [HACKERS] Re: [GSOC 17] Eliminate O(N^2) scaling from rw-conflict tracking in serializable transactions

2017-03-15 Thread Kevin Grittner
On Wed, Mar 15, 2017 at 11:35 AM, Mengxing Liu wrote: >> On a NUMA machine It is not at all unusual to see bifurcated results >> -- with each run coming in very close to one number or a second >> number, often at about a 50/50 rate, with no numbers falling >> anywhere else. This seems to be base

Re: [HACKERS] Defaulting psql to ON_ERROR_ROLLBACK=interactive

2017-03-15 Thread Joshua Yanovski
Just chiming in: I rely heavily on the current default behavior because it prevents large statements pasted into psql that cause errors in transactions from partially running, and if it were changed I would have caused production outages. On Wed, Mar 15, 2017 at 8:42 AM, Pavel Stehule wrote: > >

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
On Wed, Mar 15, 2017 at 9:28 PM, Robert Haas wrote: > On Wed, Mar 15, 2017 at 11:37 AM, Ashutosh Sharma > wrote: >>> +/* Get RelfileNode from relation OID */ >>> +rel = relation_open(htup->t_tableOid, NoLock); >>> +rnode = rel->rd_node; >>> +relation_close(rel, No

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 2:10 PM, Ashutosh Sharma wrote: > Okay, I have done the changes as suggested by you. Please refer to the > attached v8 patch. Thanks. Cool, but this doesn't look right: +action = XLogReadBufferForRedo(record, 0, &buffer); + +if (!IsBufferCleanupOK(buffer)) +

[HACKERS] Leftover invalidation handling in RemoveRelations

2017-03-15 Thread Andres Freund
Hi, reviewing some citus code copied from postgres I noticed that RemoveRelations() has the following bit: /* * These next few steps are a great deal like relation_openrv, but we * don't bother building a relcache entry since we don't need it.

Re: [HACKERS] Leftover invalidation handling in RemoveRelations

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 2:40 PM, Andres Freund wrote: > reviewing some citus code copied from postgres I noticed that > RemoveRelations() has the following bit: > > /* > * These next few steps are a great deal like > relation_openrv, but we > * do

Re: [HACKERS] Leftover invalidation handling in RemoveRelations

2017-03-15 Thread Andres Freund
Hi, On 2017-03-15 14:46:22 -0400, Robert Haas wrote: > On Wed, Mar 15, 2017 at 2:40 PM, Andres Freund wrote: > Yeah, I don't think that would hurt anything. > > (I'm not sure it'll help anything either - the overhead of an extra > AcceptInvalidationMessages() call is quite minimal - but, as you

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 8:49 AM, Ashutosh Bapat wrote: >> Of course, that supposes that 0009 can manage to postpone creating >> non-sampled child joinrels until create_partition_join_plan(), which >> it currently doesn't. > > Right. We need the child-join's RelOptInfos to estimate sizes, so that >

Re: [HACKERS] Partition-wise join for join between (declaratively) partitioned tables

2017-03-15 Thread Robert Haas
On Wed, Mar 15, 2017 at 8:55 AM, Ashutosh Bapat wrote: > Sorry. That was added by my patch to refactor > set_append_rel_pathlist(). I have added a patch in the series to > remove that line. It's not worth an extra commit just to change what isn't broken. Let's just leave it alone. >> Very sad.

Re: [HACKERS] pgbench - allow to store select results into variables

2017-03-15 Thread Fabien COELHO
Hello Rafia, I was reviewing v7 of this patch, to start with I found following white space errors when applying with git apply, /home/edb/Desktop/patches/others/pgbench-into-7.patch:66: trailing whitespace. Yep. I do not know why "git apply" sometimes complains. All is fine for me both with

Re: [HACKERS] Parallel Bitmap scans a bit broken

2017-03-15 Thread Emre Hasegeli
> Please verify the fix. The same test with both of the patches applied still crashes for me. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-15 Thread Tom Lane
Andres Freund writes: > [ latest patches ] I looked through 0002-Make-get_last_attnums-more-generic.patch. Although it seems relatively unobjectionable on its own, I'm not convinced that it's really useful to try to split it out like this. I see that 0004 removes the only call of ExecGetLastAttnu

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-15 Thread Andres Freund
On March 15, 2017 12:33:28 PM PDT, Tom Lane wrote: >Andres Freund writes: >> [ latest patches ] > >I looked through 0002-Make-get_last_attnums-more-generic.patch. >So for my money, you should drop 0002 altogether and just have 0004 >remove get_last_attnums() from execUtils.c and stick it into

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-15 Thread Tom Lane
Andres Freund writes: > On March 15, 2017 12:33:28 PM PDT, Tom Lane wrote: >> So for my money, you should drop 0002 altogether and just have 0004 >> remove get_last_attnums() from execUtils.c and stick it into >> execExpr.c. I suppose you still need the LastAttnumInfo API change >> so as to deco

Re: [HACKERS] Patch: Write Amplification Reduction Method (WARM)

2017-03-15 Thread Pavan Deolasee
On Tue, Mar 14, 2017 at 7:16 PM, Alvaro Herrera wrote: > Pavan Deolasee wrote: > > On Tue, Mar 14, 2017 at 7:17 AM, Alvaro Herrera < > alvhe...@2ndquadrant.com> > > wrote: > > > > I have already commented about the executor involvement in btrecheck(); > > > that doesn't seem good. I previously s

[HACKERS] Odd listen_addresses behavior

2017-03-15 Thread Joshua D. Drake
-hackers, I found this today: jd@jd-wks:~/snap/postgresql96/common/data$ /snap/postgresql96/19/usr/bin/pg_ctl -D data stop pg_ctl: directory "data" does not exist jd@jd-wks:~/snap/postgresql96/common/data$ cd .. jd@jd-wks:~/snap/postgresql96/common$ /snap/postgresql96/19/usr/bin/pg_ctl -D d

Re: [HACKERS] Microvacuum support for Hash Index

2017-03-15 Thread Ashutosh Sharma
> +action = XLogReadBufferForRedo(record, 0, &buffer); > + > +if (!IsBufferCleanupOK(buffer)) > +elog(PANIC, "hash_xlog_vacuum_one_page: failed to acquire > cleanup lock"); > > That could fail, I think, because of a pin from a Hot Standby backend. > You want to call XLogReadBufferFo

Re: [HACKERS] Odd listen_addresses behavior

2017-03-15 Thread Tom Lane
"Joshua D. Drake" writes: > jd@jd-wks:~/snap/postgresql96/common$ grep listen_addresses > data/postgresql.conf > listen_addresses = '192*' # what IP address(es) to listen on; > -- I wasn't actually expecting the above to work. I was just testing. Fails as expected for me: $ postgre

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-15 Thread Andres Freund
On 2017-03-15 15:41:22 -0400, Tom Lane wrote: > Andres Freund writes: > > On March 15, 2017 12:33:28 PM PDT, Tom Lane wrote: > >> So for my money, you should drop 0002 altogether and just have 0004 > >> remove get_last_attnums() from execUtils.c and stick it into > >> execExpr.c. I suppose you s

Re: [HACKERS] WIP: Faster Expression Processing v4

2017-03-15 Thread Tom Lane
Andres Freund writes: > On 2017-03-15 15:41:22 -0400, Tom Lane wrote: >> Color me dubious. Which specific other places have you got in mind, and >> do they have expression trees at hand that would tell them which columns >> they really need to pull out? > I was thinking of execGrouping.c's execT

  1   2   >