Re: Why overhead of SPI is so large?

2019-08-22 Thread Konstantin Knizhnik
On 22.08.2019 5:40, Kyotaro Horiguchi wrote: Hello. At Wed, 21 Aug 2019 19:41:08 +0300, Konstantin Knizhnik wrote in Hi, hackers. One of our customers complains about slow execution of PL/pgSQL functions comparing with Oracle. So he wants to compile PL/pgSQL functions (most likely

Take skip header out of a loop in COPY FROM

2019-08-22 Thread Surafel Temesgen
Hello, Even if skipping header is done only once its checked and skipped in a loop. If I don’t miss something it can be done out side a loop like attached patch regards Surafel diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c index 4f04d122c3..4e7709d7bf 100644 ---

Re: FETCH FIRST clause PERCENT option

2019-08-22 Thread Surafel Temesgen
On Tue, Aug 20, 2019 at 9:10 AM Kyotaro Horiguchi wrote: > Hi, > > At Wed, 7 Aug 2019 10:20:09 +0300, Surafel Temesgen > wrote in < > calay4q98xbvhtz4yj9dccmg2-s1_jurr7fyanfw+bkmr22o...@mail.gmail.com> > > Hi > > On Wed, Aug 7, 2019 at 6:11 AM Kyotaro Horiguchi < > horikyota@gmail.com> > >

Refactoring of connection with password prompt loop for frontends

2019-08-22 Thread Michael Paquier
Hi all, In six places of the code tree (+ one in psql which is a bit different), we have the following pattern for frontend tools to connect to a backend with a password prompt, roughly like that: do { [...] conn = PQconnectdbParams(keywords, values, true); [...] if

Comment in ginpostinglist.c doesn't match code

2019-08-22 Thread Heikki Linnakangas
Hi, While merging Greenplum with 9.4, we ran into problems with the GIN posting list encoding, because Greenplum sometimes uses ItemPointers with offset numbers up to 32768. The GIN posting list code was written with the assumption that the maximum is MaxHeapTuplesPerPage, and it uses only

[PATCH] Tab completion for CREATE OR REPLACE

2019-08-22 Thread Wang, Shenhao
Hello, hackers: I created a patch about tab completion for command CREATE OR REPLACE in psql includes: CREATE [ OR REPLACE ] FUNCTION CREATE [ OR REPLACE ] PROCEDURE CREATE [ OR REPLACE ] LANGUAGE CREATE [ OR REPLACE ] RULE name AS ON event CREATE [ OR

Re: Optimization of vacuum for logical replication

2019-08-22 Thread Kyotaro Horiguchi
Hello. At Wed, 21 Aug 2019 18:06:52 +0300, Konstantin Knizhnik wrote in <968fc591-51d3-fd74-8a55-40aa770ba...@postgrespro.ru> > Ok, you convinced me that there are cases when people want to combine > logical replication with streaming replication without slot. > But is it acceptable to have

Re: range_agg

2019-08-22 Thread Paul A Jungwirth
On Tue, Aug 20, 2019 at 10:33 PM Jeff Davis wrote: > > Is there any historical discussion around > > typemods on range types? > > I did find a few references: Thanks for looking those up! It's very interesting to see some of the original discussion around range types. Btw this is true of so

Re: [PATCH] Tab completion for CREATE OR REPLACE

2019-08-22 Thread Ian Barwick
On Thu, 22 Aug 2019 at 15:05, Wang, Shenhao wrote: > > Hello, hackers: > > I created a patch about tab completion for command CREATE OR REPLACE in psql > includes: > CREATE [ OR REPLACE ] FUNCTION > CREATE [ OR REPLACE ] PROCEDURE > CREATE [ OR REPLACE ] LANGUAGE >

Re: Optimization of vacuum for logical replication

2019-08-22 Thread Konstantin Knizhnik
On 22.08.2019 6:13, Kyotaro Horiguchi wrote: Hello. At Wed, 21 Aug 2019 18:06:52 +0300, Konstantin Knizhnik wrote in <968fc591-51d3-fd74-8a55-40aa770ba...@postgrespro.ru> Ok, you convinced me that there are cases when people want to combine logical replication with streaming replication

Re: Take skip header out of a loop in COPY FROM

2019-08-22 Thread Heikki Linnakangas
On 22/08/2019 11:31, Surafel Temesgen wrote: Hello, Even if skipping header is done only once its checked and skipped in a loop. If I don’t miss something it can be done out side a loop like attached patch You may be on to something, but if we move it to CopyFrom(), as in your patch, then

Re: POC: Cleaning up orphaned files using undo logs

2019-08-22 Thread Amit Kapila
On Thu, Aug 22, 2019 at 11:04 AM Dilip Kumar wrote: > > On Thu, Aug 22, 2019 at 10:24 AM Andres Freund wrote: > > > > Hi, > > > > On 2019-08-22 10:19:04 +0530, Dilip Kumar wrote: > > > On Thu, Aug 22, 2019 at 9:58 AM Andres Freund wrote: > > > > > > > > Hi, > > > > > > > > On 2019-08-22

Re: when the IndexScan reset to the next ScanKey for in operator

2019-08-22 Thread Kyotaro Horiguchi
Hi, At Wed, 21 Aug 2019 20:58:47 +0800, Alex wrote in > postgres=# select * from t2 where a in (1, 10); ... > I can see the plan stores the "1 and 10" information in > IndexScan->indexqual, which is an SCALARARRAYOPEXPR expression. ... > suppose the executor should scan 1 first, If

Re: Take skip header out of a loop in COPY FROM

2019-08-22 Thread Heikki Linnakangas
On 22/08/2019 12:54, Adam Lee wrote: My next thought is to call unlikely() here, but we don't have it... https://www.postgresql.org/message-id/CABRT9RC-AUuQL6txxsoOkLxjK1iTpyexpbizRF4Zxny1GXASGg%40mail.gmail.com We do, actually, since commit aa3ca5e3dd in v10. Not sure it's worth the trouble

Re: Make SQL/JSON error code names match SQL standard

2019-08-22 Thread Peter Eisentraut
On 2019-08-20 13:48, Alexander Korotkov wrote: > On Tue, Aug 20, 2019 at 11:49 AM Peter Eisentraut > wrote: >> I propose the attached patch to make the new SQL/JSON error code names >> match the SQL standard. The existing minor differences don't seem >> necessary. > > Thank you for noticing! >

Email to hackers for test coverage

2019-08-22 Thread movead...@highgo.ca
Hello hackers, One of the area that didn't get much attention in the community recently is analysing and increasing the code coverage of PostgreSQL regession test suite. I have started working on the code coverage by running the GCOV code coverage analysis tool in order to analyse the current

Re: Why overhead of SPI is so large?

2019-08-22 Thread Konstantin Knizhnik
On 22.08.2019 3:27, Tsunakawa, Takayuki wrote: From: Konstantin Knizhnik [mailto:k.knizh...@postgrespro.ru] PL/pgSQL: 29044.361 ms C/SPI: 22785.597 ms The fact that difference between PL/pgSQL and function implemented in C using SPI is not so large was expected by me. This

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-08-22 Thread Kyotaro Horiguchi
Hello. At Mon, 19 Aug 2019 23:03:14 -0700, Noah Misch wrote in <20190820060314.ga3086...@rfd.leadboat.com> > On Mon, Aug 19, 2019 at 06:59:59PM +0900, Kyotaro Horiguchi wrote: > > At Sat, 17 Aug 2019 20:52:30 -0700, Noah Misch wrote in > > <20190818035230.gb3021...@rfd.leadboat.com> > > > For

Re: POC: Cleaning up orphaned files using undo logs

2019-08-22 Thread Dilip Kumar
On Thu, Aug 22, 2019 at 9:21 PM Dilip Kumar wrote: > > On Thu, Aug 22, 2019 at 7:34 PM Robert Haas wrote: > > > > On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote: > > > Yeah, we can handle the bulk fetch as you suggested and it will make > > > it a lot easier. But, currently while

Re: Why overhead of SPI is so large?

2019-08-22 Thread Konstantin Knizhnik
Some more information... First of all I found out that marking PL/pgSQL function as immutable significantly increase speed of its execution: 19808 ms vs. 27594. It happens because exec_eval_simple_expr is taken snapshot if function is volatile (default). I wonder if PL/pgSQL compiler can detect

Re: Asymmetric partition-wise JOIN

2019-08-22 Thread Kohei KaiGai
Hello, Even though nobody has respond the thread, I tried to make a prototype of the asymmetric partition-wise join support. This feature tries to join non-partitioned and partitioned relation before append. See the example below: create table ptable (dist int, a int, b int) partition by hash

Re: mingw32 floating point diff

2019-08-22 Thread Tom Lane
I wrote: > Peter Eisentraut writes: >> Do we care to do anything about this? Pick slightly different test data >> perhaps? > Picking different test data might be a good "fix". Alternatively, we > could try to figure out where the discrepancy is arising and adjust > the code --- but that might

Re: POC: Cleaning up orphaned files using undo logs

2019-08-22 Thread Robert Haas
On Thu, Aug 22, 2019 at 12:54 AM Andres Freund wrote: > But why? It makes a *lot* more sense to have it in the beginning. I > don't think bulk-fetch really requires it to be in the end - we can > still process records forward on a page-by-page basis. There are two separate needs here: to be able

Re: Take skip header out of a loop in COPY FROM

2019-08-22 Thread Adam Lee
On Thu, Aug 22, 2019 at 11:48:31AM +0300, Heikki Linnakangas wrote: > On 22/08/2019 11:31, Surafel Temesgen wrote: > > Hello, > > > > Even if skipping header is done only once its checked and skipped in a > > loop. If I don’t miss something it can be done out side a loop like > > attached patch >

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-08-22 Thread Anastasia Lubennikova
22.08.2019 16:13, Paul Guo wrote: Thanks. I updated the patch to v5. It passes install-check testing and recovery testing. Hi, Thank you for working on this fix. The overall design of the latest version looks good to me. But during the review, I found a bug in the current implementation. New

Re: POC: Cleaning up orphaned files using undo logs

2019-08-22 Thread Andres Freund
Hi On August 22, 2019 9:14:10 AM PDT, Dilip Kumar wrote: > But, those requests will >ultimately be used for collecting the record by the bulk fetch. So if >we are planning to change the bulk fetch to read forward then maybe we >don't need the valid last undo record pointer because that we will

mingw32 floating point diff

2019-08-22 Thread Peter Eisentraut
On 2019-08-20 14:59, Peter Eisentraut wrote: > Running the regression tests on mingw32, I get the following diff in > circle.out: > > @@ -111,8 +111,8 @@ >WHERE (c1.f1 < c2.f1) AND ((c1.f1 <-> c2.f1) > 0) >ORDER BY distance, area(c1.f1), area(c2.f1); > five | one | two

Re: standby recovery fails (tablespace related) (tentative patch and discussion)

2019-08-22 Thread Paul Guo
Thanks. I updated the patch to v5. It passes install-check testing and recovery testing. On Fri, Aug 2, 2019 at 6:38 AM Thomas Munro wrote: > On Mon, Jul 15, 2019 at 10:52 PM Paul Guo wrote: > > Please see the attached v4 patch. > > While moving this to the next CF, I noticed that this needs

Re: Remove page-read callback from XLogReaderState.

2019-08-22 Thread Heikki Linnakangas
On 22/08/2019 04:43, Kyotaro Horiguchi wrote: At Mon, 29 Jul 2019 22:39:57 +0300, Heikki Linnakangas wrote in On 12/07/2019 10:10, Kyotaro Horiguchi wrote: * XLogReaderState->readBuf is now allocated and controlled by the * caller, not by xlogreader.c itself. When XLogReadRecord() needs

Re: mingw32 floating point diff

2019-08-22 Thread Tom Lane
Peter Eisentraut writes: > On 2019-08-20 14:59, Peter Eisentraut wrote: >> Running the regression tests on mingw32, I get the following diff in >> circle.out: ... > OK, the problem isn't the new output routines. The result of the > computations is actually different. The test itself is new in

Re: POC: Cleaning up orphaned files using undo logs

2019-08-22 Thread Robert Haas
On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote: > Yeah, we can handle the bulk fetch as you suggested and it will make > it a lot easier. But, currently while registering the undo request > (especially during the first pass) we need to compute the from_urecptr > and the to_urecptr. And, for

Fault injection framework

2019-08-22 Thread Asim R P
Hello Fault injection was discussed a few months ago at PGCon in Ottawa. At least a few folks showed interest and so I would like to present what we have been using in Greenplum. The attached patch set contains the fault injector framework ported to PostgreSQL master. It provides ability to

Re: Why overhead of SPI is so large?

2019-08-22 Thread Pavel Stehule
čt 22. 8. 2019 v 17:51 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > Some more information... > First of all I found out that marking PL/pgSQL function as immutable > significantly increase speed of its execution: > 19808 ms vs. 27594. It happens because

Re: POC: Cleaning up orphaned files using undo logs

2019-08-22 Thread Dilip Kumar
On Thu, Aug 22, 2019 at 9:55 PM Andres Freund wrote: > > Hi > > On August 22, 2019 9:14:10 AM PDT, Dilip Kumar wrote: > > But, those requests will > >ultimately be used for collecting the record by the bulk fetch. So if > >we are planning to change the bulk fetch to read forward then maybe we >

Re: POC: Cleaning up orphaned files using undo logs

2019-08-22 Thread Dilip Kumar
On Thu, Aug 22, 2019 at 7:34 PM Robert Haas wrote: > > On Thu, Aug 22, 2019 at 1:34 AM Dilip Kumar wrote: > > Yeah, we can handle the bulk fetch as you suggested and it will make > > it a lot easier. But, currently while registering the undo request > > (especially during the first pass) we

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Melanie Plageman
On Wed, Aug 21, 2019 at 12:16 PM Alvaro Herrera wrote: > On 2019-Aug-21, Melanie Plageman wrote: > > > In Greenplum, we mainly add new tests to a separate isolation > > framework (called isolation2) which uses a completely different > > syntax. It doesn't use isolationtester at all. So, I

Preserving extension ownership in dump/restore/pg_upgrade

2019-08-22 Thread Tom Lane
Currently, we fail to do $SUBJECT: post-restore, an extension will be owned by the role used to do the restore. This is because pg_dump does not consider that extensions have owners at all, so it doesn't issue ALTER EXTENSION OWNER TO, which is a command that the server doesn't have anyway.

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Robert Eckhardt
On Thu, Aug 22, 2019 at 1:45 PM Melanie Plageman wrote: > > > On Wed, Aug 21, 2019 at 12:16 PM Alvaro Herrera > wrote: >> >> On 2019-Aug-21, Melanie Plageman wrote: >> >> > In Greenplum, we mainly add new tests to a separate isolation >> > framework (called isolation2) which uses a completely

Re: Allow to_date() and to_timestamp() to accept localized names

2019-08-22 Thread Juan José Santamaría Flecha
On Sun, Aug 18, 2019 at 10:42 AM Juan José Santamaría Flecha wrote: > > Going through the current items in the wiki's todo list, I have been > looking into: "Allow to_date () and to_timestamp () to accept > localized month names". > I have gone through a second take on this, trying to give it a

Infinite wait for SyncRep while handling USR1

2019-08-22 Thread Soumyadeep Chakraborty
Hello Hackers, There is an edge case in 9_5_STABLE (doesn't reproduce 9_6+) we found in how backends handle the TERM signal while handling a USR1 signal that can cause them to infinitely wait. A backend can end up waiting forever inside SyncRepWaitForLSN() at: rc = WaitLatch(MyLatch,

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Michael Paquier
On Thu, Aug 22, 2019 at 10:20:48AM -0700, Melanie Plageman wrote: > So, there is some historical context as to why it is a separate test suite. > And some of the differences are specific to Greenplum -- e.g. needing to > connect to a specific database in "utility mode" to do something. What is

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2019-08-22 Thread Michael Paquier
On Fri, Aug 23, 2019 at 11:09:44AM +0900, Masahiko Sawada wrote: > While updating the doc I realized that > perhaps we should have the new section for heap and put the > descriptions of heap functions into it rather than having them as > general functions. If we need this change it is for PG12. I

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Melanie Plageman
On Thu, Aug 22, 2019 at 6:53 PM Michael Paquier wrote: > On Thu, Aug 22, 2019 at 10:20:48AM -0700, Melanie Plageman wrote: > > So, there is some historical context as to why it is a separate test > suite. > > And some of the differences are specific to Greenplum -- e.g. needing to > > connect to

Hooks for session start and end, take two

2019-08-22 Thread Michael Paquier
Hi all, Attached is a patch set to respawn the issue of $subject which has been discussed here: https://www.postgresql.org/message-id/20170720204733.40f2b7eb.nag...@sraoss.co.jp The patch has been committed once as of cd8ce3a but it got shortly reverted after with 98d54bb because of buildfarm

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2019-08-22 Thread Peter Geoghegan
On Wed, Aug 21, 2019 at 10:19 AM Anastasia Lubennikova wrote: > I'm going to look through the patch once more to update nbtxlog > comments, where needed and > answer to your remarks that are still left in the comments. Have you been using amcheck's rootdescend verification? I see this problem

Re: Cleanup isolation specs from unused steps

2019-08-22 Thread Michael Paquier
On Thu, Aug 22, 2019 at 09:19:47PM -0700, Melanie Plageman wrote: > It's a connection parameter that allows you to connect to a single Postgres > node in a Greenplum cluster. I only included it as an example of the kind of > "Greenplum-specific" things that are in the test framework. Ah, I see.

Re: when the IndexScan reset to the next ScanKey for in operator

2019-08-22 Thread Peter Geoghegan
On Wed, Aug 21, 2019 at 6:24 AM Alex wrote: > suppose the executor should scan 1 first, If all the tuples for 1 has been > scanned, then **it should be reset to 10** and scan again. You might find my nbtree index scan test patch useful:

Re: [HACKERS] [PATCH] pageinspect function to decode infomasks

2019-08-22 Thread Masahiko Sawada
On Thu, Aug 22, 2019 at 12:36 AM Masahiko Sawada wrote: > > On Thu, Aug 22, 2019 at 12:19 AM Alvaro Herrera > wrote: > > > > Can I interest someone into updating this patch? We now have (I think) > > an agreed design, and I think the development work needed should be > > straightforward. We

Re: Comment in ginpostinglist.c doesn't match code

2019-08-22 Thread Ashwin Agrawal
On Thu, Aug 22, 2019 at 1:14 AM Heikki Linnakangas wrote: > > The patch also includes a little unit test module to test this without > creating a 16 TB table. A whole new test module seems a bit like > overkill just for this, but clearly we were missing test coverage here. > And it will come