Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Michael Paquier
On Tue, Aug 20, 2019 at 12:34:45AM -0400, Alvaro Herrera wrote: > I created the dry-run mode to be able to easily generate the set of > possible permutations for a new test, then edit the result and put it > back in the spec file; but after the deadlock tests were added (with > necessary hacking

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Alvaro Herrera
On 2019-Aug-20, Michael Paquier wrote: > On Mon, Aug 19, 2019 at 10:23:19AM -0700, Melanie Plageman wrote: > > Could you do the check that all steps have been used in dry_run mode > > instead of when running the tests for real? > > Sure, I was hesitating to do so. I have no issue in moving the

Fixing typos and inconsistencies

2019-08-19 Thread Alexander Lakhin
Hello hackers, Now that the unicums checking is finished, I would like to share the script I used to find them. Maybe it can be useful to recheck the source tree from time to time... I don't think that the check could be fully automated, but with some eyeballing it allows to maintain a more

Re: Zedstore - compressed in-core columnar storage

2019-08-19 Thread Justin Pryzby
On Mon, Aug 19, 2019 at 04:15:30PM -0700, Alexandra Wang wrote: > On Sun, Aug 18, 2019 at 12:35 PM Justin Pryzby wrote: > > > . I was missing a way to check for compression ratio; > > Here are the ways to check compression ratio for zedstore: > > Table level: > SELECT

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Michael Paquier
On Mon, Aug 19, 2019 at 10:23:19AM -0700, Melanie Plageman wrote: > Could you do the check that all steps have been used in dry_run mode > instead of when running the tests for real? Sure, I was hesitating to do so. I have no issue in moving the check into run_testspec(). So done as attached.

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Michael Paquier
On Mon, Aug 19, 2019 at 11:02:42AM -0400, Tom Lane wrote: > Michael Paquier writes: >> I have been looking at the isolation tests, and we have in some specs >> steps which are defined but not used in any permutations. > > Hmm, might any of those represent actual bugs? Or are they just >

Plug-in common/logging.h with vacuumlo and oid2name

2019-08-19 Thread Michael Paquier
Hi all, While refactoring some code, I have bumped into $subject, which causes both tools to have incorrect error message formats and progname being used all through the code, sometimes incorrectly or just missing. At the same time, we are missing a call to set_pglocale_pgservice() for both

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2019-08-19 Thread Peter Geoghegan
On Fri, Aug 16, 2019 at 8:56 AM Anastasia Lubennikova wrote: > Now the algorithm is the following: > > - If bt_findinsertloc() found out that tuple belongs to existing posting > tuple's > TID interval, it sets 'in_posting_offset' variable and passes it to > _bt_insertonpg() > > - If

Re: [HACKERS] [WIP] Effective storage of duplicates in B-tree index.

2019-08-19 Thread Peter Geoghegan
On Tue, Aug 13, 2019 at 8:45 AM Anastasia Lubennikova wrote: > > I still need to think about the exact details of alignment within > > _bt_insertonpg_in_posting(). I'm worried about boundary cases there. I > > could be wrong. > Could you explain more about these cases? > Now I don't understand

Re: Zedstore - compressed in-core columnar storage

2019-08-19 Thread Alexandra Wang
On Sun, Aug 18, 2019 at 12:35 PM Justin Pryzby wrote: > > . I was missing a way to check for compression ratio; Here are the ways to check compression ratio for zedstore: Table level: select sum(uncompressedsz::numeric) / sum(totalsz) as compratio from pg_zs_btree_pages(); Per column level:

Re: PANIC: could not flush dirty data: Operation not permitted power8, Redhat Centos

2019-08-19 Thread Thomas Munro
On Mon, Aug 19, 2019 at 7:32 AM Thomas Munro wrote: > On Wed, Apr 17, 2019 at 1:04 PM Thomas Munro wrote: > > On Mon, Apr 15, 2019 at 7:57 PM wrote: > > > I forgot to mention that this is happening in a docker container. > > > > Huh, so there may be some configuration of Linux container that

Re: Unused header file inclusion

2019-08-19 Thread Tom Lane
Andres Freund writes: > On 2019-08-19 13:07:50 -0700, Andres Freund wrote: >> On 2019-08-18 14:37:34 -0400, Tom Lane wrote: >>> Oddly, cpluspluscheck does not complain about those cases, but it >>> does complain about >> Hm. I don't understand why it's not complaining. Wonder if it's a >>

Re: Unused header file inclusion

2019-08-19 Thread Alvaro Herrera
On 2019-Aug-19, Andres Freund wrote: > > I wish we could move the whole logic of those scripts into makefiles, so > > we could employ parallelism. > > Hm. Perhaps the way to do that would be to use gcc's -include to include > postgres.h, and use -Wc++-compat to detect c++ issues, rather than

Re: POC: Cleaning up orphaned files using undo logs

2019-08-19 Thread Andres Freund
On 2019-08-19 17:52:24 +0530, Amit Kapila wrote: > On Sat, Aug 17, 2019 at 10:58 PM Andres Freund wrote: > > > > Hi, > > > > On 2019-08-17 12:05:21 -0400, Robert Haas wrote: > > > On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote: > > > > > > Again, I think it's not ok to just assume you can

Re: Unused header file inclusion

2019-08-19 Thread Andres Freund
Hi, On 2019-08-19 13:07:50 -0700, Andres Freund wrote: > On 2019-08-18 14:37:34 -0400, Tom Lane wrote: > > That's with a script I use that's like cpluspluscheck except it tests > > with plain gcc not g++. I attached it for the archives' sake. > > > > Oddly, cpluspluscheck does not complain

Re: io_uring support

2019-08-19 Thread Andres Freund
Hi, On 2019-08-19 20:20:46 +0200, Dmitry Dolgov wrote: > For already some time I'm following the new linux IO interface "io_uring", > that > was introduced relatively recently [1]. Short description says: > > Shared application/kernel submission and completion ring pairs, for >

Re: Unused header file inclusion

2019-08-19 Thread Andres Freund
Hi, On 2019-08-18 14:37:34 -0400, Tom Lane wrote: > Andres Freund writes: > > I've pushed the other ones. > > Checking whether header files compile standalone shows you were overly > aggressive about removing fmgr.h includes: > > In file included from /tmp/headerscheck.Ss8bVx/test.c:3: >

io_uring support

2019-08-19 Thread Dmitry Dolgov
Hi, For already some time I'm following the new linux IO interface "io_uring", that was introduced relatively recently [1]. Short description says: Shared application/kernel submission and completion ring pairs, for supporting fast/efficient IO. For us the important part is probably

Re: Unused header file inclusion

2019-08-19 Thread Tom Lane
Alvaro Herrera writes: > On 2019-Aug-18, Tom Lane wrote: >> I did that, and ended up with the attached. I'm rather tempted to stick >> this into src/tools/ alongside cpluspluscheck, because it seems to find >> rather different trouble spots than cpluspluscheck does. Thoughts? > Yeah, let's

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Melanie Plageman
On Mon, Aug 19, 2019 at 1:08 AM Michael Paquier wrote: > Hi all, > > I have been looking at the isolation tests, and we have in some specs > steps which are defined but not used in any permutations. In order to > detect them, I have been using the attached trick to track which > permutations

Re: Global temporary tables

2019-08-19 Thread Pavel Stehule
> Certainly, default (small) temp buffer size plays roles. > But it this IPC host this difference is not so important. > Result with local temp tables and temp_buffers = 1GB: 859k TPS. > It is little bit unexpected result.I understand so it partially it is generic problem access to smaller

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-19 Thread Ahsan Hadi
I have shared a calendar invite for TDE/KMS weekly meeting with the members who expressed interest of joining the meeting in this chain. Hopefully I haven't missed anyone. I am not aware of everyone's timezone but I have tried to setup a time that's not very inconvenient. It won't be ideal for

Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)

2019-08-19 Thread Joe Conway
On 8/19/19 8:51 AM, Ahsan Hadi wrote: > I have shared a calendar invite for TDE/KMS weekly meeting with the > members who expressed interest of joining the meeting in this chain. > Hopefully I haven't missed anyone. > > I am not aware of everyone's timezone but I have tried to setup a time >

Re: Duplicated LSN in ReorderBuffer

2019-08-19 Thread Andres Freund
Hi, On August 19, 2019 7:43:28 AM PDT, Alvaro Herrera wrote: >On 2019-Aug-19, Masahiko Sawada wrote: > >> On Tue, Aug 13, 2019 at 6:36 AM Alvaro Herrera > wrote: > >> > BTW I wrote the code as suggested and it passes all the tests ... >but I >> > then noticed that the unpatched code doesn't

Re: Partial join

2019-08-19 Thread Arne Roland
Richard Guo wrote: > Please refer to the code snippet below: > > @@ -1164,8 +1164,8 @@ generate_join_implied_equalities(PlannerInfo *root, > List *sublist = NIL; > > /* ECs containing consts do not need any further enforcement > */ > if

Re: Cleanup isolation specs from unused steps

2019-08-19 Thread Tom Lane
Michael Paquier writes: > I have been looking at the isolation tests, and we have in some specs > steps which are defined but not used in any permutations. Hmm, might any of those represent actual bugs? Or are they just leftovers from test development? > In order to > detect them, I have been

Wrong sentence in the README?

2019-08-19 Thread Daniel Westermann (DWE)
Hi, in the README, top level, there is this: PostgreSQL has many language interfaces, many of which are listed here: https://www.postgresql.org/download I don't think the download page lists any language interfaces or do I miss something? Regards Daniel

Re: Duplicated LSN in ReorderBuffer

2019-08-19 Thread Alvaro Herrera
On 2019-Aug-19, Masahiko Sawada wrote: > On Tue, Aug 13, 2019 at 6:36 AM Alvaro Herrera > wrote: > > BTW I wrote the code as suggested and it passes all the tests ... but I > > then noticed that the unpatched code doesn't fail Ildar's original > > pgbench-based test for me, either. So maybe

Re: psql \r command is working?

2019-08-19 Thread Sergei Kornilov
so this is expected behavior, ok, thank you! regards, Sergei

Re: psql \r command is working?

2019-08-19 Thread Tom Lane
Sergei Kornilov writes: > Is psql command \r actually works? I expected \r and then \e command should > start editor without text of last query. \r clears the current query buffer. \e without an argument is defined to edit the current query buffer, or if that is empty, the last query. I don't

Re: Unused header file inclusion

2019-08-19 Thread Alvaro Herrera
On 2019-Aug-18, Tom Lane wrote: > I wrote: > > (My headerscheck script is missing that header; I need to update it to > > match the latest version of cpluspluscheck.) > > I did that, and ended up with the attached. I'm rather tempted to stick > this into src/tools/ alongside cpluspluscheck,

psql \r command is working?

2019-08-19 Thread Sergei Kornilov
Hello Is psql command \r actually works? I expected \r and then \e command should start editor without text of last query. LANG=C EDITOR=cat psql psql (11.5 (Debian 11.5-1.pgdg100+1)) Type "help" for help. melkij=> select 1; ?column? -- 1 (1 row) melkij=> \r Query buffer

Re: (select query)/relation as first class citizen

2019-08-19 Thread Roman Pekar
Hi, John, I think you've outlined the problem and possible solutions quite well. It's great to see that the goal might be not that far from implementing.

Re: POC: Cleaning up orphaned files using undo logs

2019-08-19 Thread Amit Kapila
On Sat, Aug 17, 2019 at 10:58 PM Andres Freund wrote: > > Hi, > > On 2019-08-17 12:05:21 -0400, Robert Haas wrote: > > On Wed, Aug 14, 2019 at 12:39 PM Andres Freund wrote: > > > > > Again, I think it's not ok to just assume you can lock an essentially > > > > > unbounded number of buffers. This

Re: Global temporary tables

2019-08-19 Thread Konstantin Knizhnik
On 19.08.2019 14:25, Pavel Stehule wrote: po 19. 8. 2019 v 13:16 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> napsal: On 19.08.2019 11:51, Konstantin Knizhnik wrote: On 18.08.2019 11:28, Pavel Stehule wrote: ne 18. 8. 2019 v 9:02 odesílatel

Re: FETCH FIRST clause PERCENT option

2019-08-19 Thread Surafel Temesgen
On Mon, Aug 19, 2019 at 1:55 PM Erik Rijkers wrote: > Another little thing, not sure it's a bug: > > limit interprets its argument by rounding up or down as one would > expect: > > table onek limit 10.4; --> gives 10 rows > table onek limit 10.6; --> gives 11 rows > > but FETCH count PERCENT

Re: Global temporary tables

2019-08-19 Thread Pavel Stehule
po 19. 8. 2019 v 13:16 odesílatel Konstantin Knizhnik < k.knizh...@postgrespro.ru> napsal: > > > On 19.08.2019 11:51, Konstantin Knizhnik wrote: > > > > On 18.08.2019 11:28, Pavel Stehule wrote: > > > > ne 18. 8. 2019 v 9:02 odesílatel Konstantin Knizhnik < > k.knizh...@postgrespro.ru> napsal: >

Re: Global temporary tables

2019-08-19 Thread Konstantin Knizhnik
On 19.08.2019 11:51, Konstantin Knizhnik wrote: On 18.08.2019 11:28, Pavel Stehule wrote: ne 18. 8. 2019 v 9:02 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> napsal: On 16.08.2019 20:17, Pavel Stehule wrote: pá 16. 8. 2019 v 16:12 odesílatel Konstantin

Re: [HACKERS] WAL logging problem in 9.4.3?

2019-08-19 Thread Kyotaro Horiguchi
Thank you for taking time. At Sat, 17 Aug 2019 20:52:30 -0700, Noah Misch wrote in <20190818035230.gb3021...@rfd.leadboat.com> > For two-phase commit, PrepareTransaction() needs to execute pending syncs. Now TwoPhaseFileHeader has two new members for (commit-time) pending syncs. Pending-syncs

Re: Rearranging ALTER TABLE to avoid multi-operations bugs

2019-08-19 Thread movead li
> This review seems not very on-point, because I made no claim to have fixed > any of those bugs. The issue at the moment is how to structure the code I am sorry for that and I have another question now. I researched the related code and find something as below: Code:

Re: FETCH FIRST clause PERCENT option

2019-08-19 Thread Erik Rijkers
On 2019-08-19 11:18, Surafel Temesgen wrote: [..] [percent-incremental-v7.patch] Thanks. Another little thing, not sure it's a bug: limit interprets its argument by rounding up or down as one would expect: table onek limit 10.4; --> gives 10 rows table onek limit 10.6; --> gives 11

Re: FETCH FIRST clause PERCENT option

2019-08-19 Thread Surafel Temesgen
Hi Erik On Mon, Aug 19, 2019 at 10:47 AM Erik Rijkers wrote: > Hi, > > (running with those two patches applied) > >select * from onek where thousand < 5 order by thousand fetch first -1 > percent rows only > is correctly caught (with "ERROR: PERCENT must not be negative") but: > >select

Re: Duplicated LSN in ReorderBuffer

2019-08-19 Thread Masahiko Sawada
On Tue, Aug 13, 2019 at 6:36 AM Alvaro Herrera wrote: > > On 2019-Aug-07, Andres Freund wrote: > > > I think we would need to do this for all values of > > SnapBuildCurrentState() - after all the problem occurs because we > > *previously* didn't assign subxids to the toplevel xid. Compared to

Re: Global temporary tables

2019-08-19 Thread Konstantin Knizhnik
On 18.08.2019 11:28, Pavel Stehule wrote: ne 18. 8. 2019 v 9:02 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>> napsal: On 16.08.2019 20:17, Pavel Stehule wrote: pá 16. 8. 2019 v 16:12 odesílatel Konstantin Knizhnik mailto:k.knizh...@postgrespro.ru>>

Re: pgsql: doc: Add some images

2019-08-19 Thread Peter Eisentraut
On 2019-08-16 22:00, Alvaro Herrera wrote: > Now when I test Jürgen's new proposed image genetic-algorithm I find > that this stuff doesn't work in VPATH builds, at least for PDF -- I > don't get a build failure, but instead I get just a section title that > doesn't precede any actual image.

Cleanup isolation specs from unused steps

2019-08-19 Thread Michael Paquier
Hi all, I have been looking at the isolation tests, and we have in some specs steps which are defined but not used in any permutations. In order to detect them, I have been using the attached trick to track which permutations are used. This allows to find immediately any over-engineered spec by

Re: FETCH FIRST clause PERCENT option

2019-08-19 Thread Erik Rijkers
On 2019-08-19 01:33, Ryan Lambert wrote: The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:tested, passed The latest patch [1]

Re: Fix typos and inconsistencies for HEAD (take 11)

2019-08-19 Thread Michael Paquier
On Mon, Aug 19, 2019 at 04:22:44PM +0900, Michael Paquier wrote: > On Mon, Aug 19, 2019 at 07:04:04AM +0300, Alexander Lakhin wrote: >> 11.57 PASSBYVALUE -> PASSEDBYVALUE > > Will fix this one separately and back-patch. No need to, actually, as the error comes from 7bdc655. -- Michael

Re: Fix typos and inconsistencies for HEAD (take 11)

2019-08-19 Thread Michael Paquier
On Mon, Aug 19, 2019 at 07:04:04AM +0300, Alexander Lakhin wrote: > 11.23 TupleLockUpdate -> LockTupleNoKeyExclusive Not sure about this one, so discarded for now. Alvaro? > 11.25 typstore -> tupstore This one is cute. It actually does not cause a compilation failure as

Re: [PATCH] minor doc fix for create-role

2019-08-19 Thread Tara Anne
Hi Tom, I agree mostly. It actually does have the words “SQL identifier” in the patch. But you are right it doesn’t link to what a SQL identifier is, but it does provide a practical solution of quoting. That was the part I cared about as a user, I just wanted to solve my problem of an email

Re: Patch: New GUC prepared_statement_limit to limit memory used by prepared statements

2019-08-19 Thread Daniel Migowski
Am 19.08.2019 um 05:57 schrieb Kyotaro Horiguchi: At Sun, 18 Aug 2019 09:43:09 +0200, Daniel Migowski wrote in <6e25ca12-9484-8994-a1ee-40fdbe6af...@ikoffice.de> Am 17.08.2019 um 19:10 schrieb Ibrar Ahmed: On Sat, Aug 17, 2019 at 6:58 PM Daniel Migowski mailto:dmigow...@ikoffice.de>> wrote:

Re: POC: Cleaning up orphaned files using undo logs

2019-08-19 Thread Dilip Kumar
On Wed, Aug 14, 2019 at 10:35 PM Andres Freund wrote: > > > > > - When reading an undo record, the whole stage of UnpackUndoData() > > > reading data into a the UndoPackContext is omitted, reading directly > > > into the UnpackedUndoRecord. That removes one further copy of the > > >