Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-12-12 Thread Andres Freund
On 2017-12-13 16:02:45 +0900, Masahiko Sawada wrote: > When we add extra blocks on a relation do we access to the disk? I > guess we just call lseek and write and don't access to the disk. If so > the performance degradation regression might not be much. Usually changes in the file size require

Top-N sorts verses parallelism

2017-12-12 Thread Thomas Munro
Hi hackers, The start-up cost of bounded (top-N) sorts is sensitive at the small end of N, and the comparison_cost * tuples * LOG2(2.0 * output_tuples) curve doesn't seem to correspond to reality. Here's a contrived example that comes from a benchmark: create table foo as select

Re: Parallel Index Scan vs BTP_DELETED and BTP_HALF_DEAD

2017-12-12 Thread Kuntal Ghosh
On Tue, Dec 12, 2017 at 5:20 PM, Amit Kapila wrote: > On Tue, Dec 12, 2017 at 4:00 PM, Kuntal Ghosh > wrote: >> On Mon, Dec 11, 2017 at 2:26 PM, Thomas Munro >> wrote: >>> On Mon, Dec 11, 2017 at 8:14 PM, Amit

Re: [HACKERS] SERIALIZABLE with parallel query

2017-12-12 Thread Thomas Munro
On Wed, Dec 13, 2017 at 2:09 PM, Haribabu Kommi wrote: > Thanks for explaining the problem in generating an isolation test to > test the serialize parallel query. > > Committer can decide whether existing test is fine to part of the test suite > or remove it, other than

Re: Parallel Index Scan vs BTP_DELETED and BTP_HALF_DEAD

2017-12-12 Thread Thomas Munro
On Wed, Dec 13, 2017 at 4:20 PM, Thomas Munro wrote: > On Wed, Dec 13, 2017 at 3:41 PM, Amit Kapila wrote: >> Good. I hope that the patch I have posted above is able to resolve >> this problem. I am asking as you haven't explicitly

pg_ctl on windows can't open postmaster.pid: Permission denied

2017-12-12 Thread Andres Freund
Hi, Buildfarm animal thrips just failed with a curious error: https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=thrips=2017-12-13%2002%3A27%3A27 == shutting down postmaster == pg_ctl: could not open PID file

Re: [HACKERS] Refactor handling of database attributes between pg_dump and pg_dumpall

2017-12-12 Thread Haribabu Kommi
On Wed, Nov 29, 2017 at 4:25 PM, Michael Paquier wrote: > On Wed, Nov 8, 2017 at 8:50 AM, Haribabu Kommi > wrote: > > Ok. Removed the documentation changes that it cannot be used for normal > > scenarios, and also added a Note section

Re: Incorrect debug info printed in generate_partition_wise_join_paths

2017-12-12 Thread Etsuro Fujita
(2017/12/13 0:56), Robert Haas wrote: Committed. Thanks! Best regards, Etsuro Fujita

Re: [HACKERS] pg_upgrade failed with error - ERROR: column "a" in child table must be marked NOT NULL

2017-12-12 Thread Michael Paquier
On Wed, Dec 13, 2017 at 10:41 AM, Amit Langote wrote: > On 2017/12/13 9:22, Ali Akbar wrote: >> For me, it's better to prevent that from happening. So, attempts to >> DROP NOT NULL on the child must be rejected. The attached patch does >> that. >> >> Unfortunately,

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread David G. Johnston
On Tue, Dec 12, 2017 at 6:26 PM, Craig Ringer wrote: > >> There is also stackexchange question with 51000 views that asks how to >> start the postgresql client. Should we rename psql to mysql to help >> new users too? >> > > Frankly, that's a legit question and I don't

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-12-12 Thread Masahiko Sawada
On Wed, Dec 13, 2017 at 12:03 AM, Robert Haas wrote: > On Mon, Dec 11, 2017 at 5:20 AM, Masahiko Sawada > wrote: >>> The question I have is how would we deal with a foreign server that is >>> not available for longer duration due to crash, longer

[PATCH] Possible NULL deref in str_time

2017-12-12 Thread Bill Moyers
I noticed that the output of pg_localtime() in str_time() is not checked and can sometimes return NULL. It's pretty unlikely to occur, I guess if the time() function was acting funny. For example if I define this: time_t fake_time(void *blah) { return 0x0110; } and then call

Re: [HACKERS] pg_upgrade failed with error - ERROR: column "a" in child table must be marked NOT NULL

2017-12-12 Thread Amit Langote
Hi. On 2017/12/13 9:22, Ali Akbar wrote: > For me, it's better to prevent that from happening. So, attempts to > DROP NOT NULL on the child must be rejected. The attached patch does > that. > > Unfortunately, pg_class has no "has_parent" attribute, so in this > patch, it hits pg_inherits

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Craig Ringer
On 12 December 2017 at 23:28, Avinash Kumar wrote: > It takes a few hours to learn PostgreSQL and 1 second to remember, how to > quit from psql. > Surprised to see the depth of discussion on this. > > It takes sometime for a person to explore or know about PostgreSQL

Re: Parallel Index Scan vs BTP_DELETED and BTP_HALF_DEAD

2017-12-12 Thread Thomas Munro
Hi, Here's a reproducer which enabled me to reach this stuck state: pid | wait_event |query ---+---+- 64617 | | select pid, wait_event, query from

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Craig Ringer
On 12 December 2017 at 23:02, Geoff Winkless wrote: > I wouldn't exactly say -1 but it's a "meh" from me. > > On 8 December 2017 at 13:57, Daniel Vérité" > wrote: > > "How to exit from PostgreSQL command line utility: psql" > > > > now at 430k views

Re: [HACKERS] SERIALIZABLE with parallel query

2017-12-12 Thread Haribabu Kommi
On Fri, Dec 8, 2017 at 11:33 AM, Thomas Munro wrote: > On Thu, Nov 30, 2017 at 2:44 PM, Thomas Munro > wrote: > > On Thu, Nov 30, 2017 at 2:32 PM, Michael Paquier > > wrote: > >> Could this question be

Re: [HACKERS] Custom compression methods

2017-12-12 Thread Robert Haas
On Tue, Dec 12, 2017 at 5:07 PM, Tomas Vondra wrote: >> I definitely think there's a place for compression built right into >> the data type. I'm still happy about commit >> 145343534c153d1e6c3cff1fa1855787684d9a38 -- although really, more >> needs to be done there.

Re: Leftover reference to replacement selection 1 run case

2017-12-12 Thread Robert Haas
On Tue, Dec 12, 2017 at 6:44 PM, Peter Geoghegan wrote: > Yes, you have that right. Per dumptuples(), even the zero tuple run edge > case will still write a run marker, and will therefore still consume a tape. > We must have at least two initial runs to merge. (though dummy runs for

Re: [HACKERS] pg_upgrade failed with error - ERROR: column "a" in child table must be marked NOT NULL

2017-12-12 Thread Ali Akbar
> For me, it's better to prevent that from happening. So, attempts to > DROP NOT NULL on the child must be rejected. The attached patch does > that. I'm sorry. Accidentaly i "--color"-ed the patch format. Attached the correct patch. Best Regards, Ali Akbar diff --git

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Michael Paquier
On Wed, Dec 13, 2017 at 8:30 AM, Andres Freund wrote: > On 2017-12-13 08:27:42 +0900, Michael Paquier wrote: >> >> (OTOH, I bet we could drop reltime/abstime without too many complaints. >> >> Y2038 is coming.) >> > >> > I'm actually about to send a patch doing so, that code

Re: Leftover reference to replacement selection 1 run case

2017-12-12 Thread Peter Geoghegan
Yes, you have that right. Per dumptuples(), even the zero tuple run edge case will still write a run marker, and will therefore still consume a tape. We must have at least two initial runs to merge. (though dummy runs for non final merges are a slightly different matter.) -- Peter Geoghegan (Sent

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread David Steele
On 12/12/17 6:33 PM, Andres Freund wrote: On 2017-12-12 18:30:47 -0500, David Steele wrote: If we had a way to prevent relfilenode reuse across multiple checkpoints this'd be easier, although ALTER TABLE SET UNLOGGED still'd complicate. Or error the backup if there is wraparound? That

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread Andres Freund
Hi, On 2017-12-12 18:30:47 -0500, David Steele wrote: > > If we had a way to prevent relfilenode reuse across multiple checkpoints > > this'd be easier, although ALTER TABLE SET UNLOGGED still'd complicate. > > Or error the backup if there is wraparound? That seems entirely unacceptable to me.

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread David Steele
On 12/12/17 6:21 PM, Andres Freund wrote: On 2017-12-12 18:18:09 -0500, David Steele wrote: On 12/12/17 6:07 PM, Andres Freund wrote: It's quite different - in the recovery case there's no other write activity going on. But on a normally running cluster the persistence of existing tables can

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Andres Freund
On 2017-12-13 08:27:42 +0900, Michael Paquier wrote: > On Wed, Dec 13, 2017 at 6:50 AM, Andres Freund wrote: > > On 2017-12-12 16:47:17 -0500, Tom Lane wrote: > >> Really? We've got test cases that intentionally exercise overflow > >> in the money code? I think we could just

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Michael Paquier
On Wed, Dec 13, 2017 at 6:50 AM, Andres Freund wrote: > On 2017-12-12 16:47:17 -0500, Tom Lane wrote: >> Really? We've got test cases that intentionally exercise overflow >> in the money code? I think we could just drop such tests, until >> such time as someone fixes the

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread David Steele
Hi Michael, On 12/12/17 6:08 PM, Michael Paquier wrote: If the forks are written out of order (i.e. main before init), which is definitely possible, then I think worst case is some files will be backed up that don't need to be. The main fork is unlikely to be very large at that point so it

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread Andres Freund
On 2017-12-12 18:18:09 -0500, David Steele wrote: > On 12/12/17 6:07 PM, Andres Freund wrote: > > > > > > I don't see this as any different than what happens during recovery. The > > > unlogged forks are cleaned / re-inited before replay starts which is the > > > same thing we are doing here. > >

Re: Learned Indexes in PostgreSQL?

2017-12-12 Thread Stefan Keller
2017-12-12 21:26 GMT+01:00 Robert Haas : > Here are links to the other two threads: > > http://postgr.es/m/CANNMO+J1KeTSx5q5SYuwHf1v-gPRLrOZw1s7qOpqWx=3umm...@mail.gmail.com > http://postgr.es/m/caaerrx8bhiw3rgaqpolqjhyhk7ghordtke5uc9dasv1w5fz...@mail.gmail.com Thanks,

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread Andres Freund
Hi, On 2017-12-12 18:04:44 -0500, David Steele wrote: > On 12/12/17 5:52 PM, Andres Freund wrote: > > On 2017-12-12 17:49:54 -0500, David Steele wrote: > > > Including unlogged relations in base backups takes up space and is > > > wasteful > > > since they are truncated during backup recovery. >

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread David Steele
Hi Andres, On 12/12/17 5:52 PM, Andres Freund wrote: On 2017-12-12 17:49:54 -0500, David Steele wrote: Including unlogged relations in base backups takes up space and is wasteful since they are truncated during backup recovery. The attached patches exclude unlogged relations from base backups

Re: PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread Andres Freund
Hi, On 2017-12-12 17:49:54 -0500, David Steele wrote: > Including unlogged relations in base backups takes up space and is wasteful > since they are truncated during backup recovery. > > The attached patches exclude unlogged relations from base backups except for > the init fork, which is

PATCH: Exclude unlogged tables from base backups

2017-12-12 Thread David Steele
Including unlogged relations in base backups takes up space and is wasteful since they are truncated during backup recovery. The attached patches exclude unlogged relations from base backups except for the init fork, which is required to recreate the main fork during recovery. *

Re: [HACKERS] Custom compression methods

2017-12-12 Thread Chapman Flack
On 12/12/2017 04:33 PM, Robert Haas wrote: > you want to upgrade your existing tables to use lz4 rather than pglz, > you can change the compression option for those columns to COMPRESS > lz4 PRESERVE pglz if you want to do it incrementally or just COMPRESS This is a thread I've only been

Re: [HACKERS] Custom compression methods

2017-12-12 Thread Tomas Vondra
On 12/12/2017 10:33 PM, Robert Haas wrote: > On Mon, Dec 11, 2017 at 2:53 PM, Tomas Vondra > wrote: >> But let me play the devil's advocate for a while and question the >> usefulness of this approach to compression. Some of the questions were >> mentioned in the

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Chapman Flack
On 12/12/2017 03:39 PM, Robert Haas wrote: > On Tue, Dec 12, 2017 at 10:02 AM, Geoff Winkless >> Ctrl-D works almost everywhere. It takes 2 seconds... > > If it really took everyone 2 seconds, this question would not be as > common as it is. ... Some years ago, I took a

Re: explain analyze output with parallel workers - question about meaning of information for explain.depesz.com

2017-12-12 Thread Robert Haas
On Sat, Dec 9, 2017 at 5:30 AM, Amit Kapila wrote: > On Sat, Dec 9, 2017 at 1:24 AM, Robert Haas wrote: >> On Fri, Dec 8, 2017 at 5:11 AM, Amit Kapila wrote: Okay, I have adjusted the patch accordingly. I have also

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Andres Freund
On 2017-12-12 16:47:17 -0500, Tom Lane wrote: > Really? We've got test cases that intentionally exercise overflow > in the money code? I think we could just drop such tests, until > such time as someone fixes the issue. Some parts at least (input & output), I think it's easy enough to fix those

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Tom Lane
Andres Freund writes: >> Robert Haas writes: >>> Long story short, I don't think anyone cares about this enough to >>> spend effort fixing it. I suspect the money data type has very few >>> users. > I'm unfortunately not so sure :(. There seem to be

Re: plpgsql test layout

2017-12-12 Thread Tom Lane
Peter Eisentraut writes: > On 12/11/17 19:29, Michael Paquier wrote: >> If I read vcregress.pl correctly, it seems to me that you need to do >> more with MSVC (see plcheck). The tests would kick if sql/ and >> expected/ are found, and the test list is fetched by

Re: CUBE seems a bit confused about ORDER BY

2017-12-12 Thread Tomas Vondra
On 12/12/2017 01:52 PM, Alexander Korotkov wrote: > On Tue, Dec 12, 2017 at 3:49 PM, Teodor Sigaev > wrote: > > Yes, the thing is that we change behavior of existing ~> > operator.  In general, this is not good idea because it could >

Re: [HACKERS] Custom compression methods

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 2:53 PM, Tomas Vondra wrote: > But let me play the devil's advocate for a while and question the > usefulness of this approach to compression. Some of the questions were > mentioned in the thread before, but I don't think they got the

Re: Rethinking MemoryContext creation

2017-12-12 Thread Tom Lane
Andres Freund writes: > On 2017-12-12 14:50:37 -0500, Robert Haas wrote: >> It strikes me that a way to optimize these cases even more would be to >> postpone creating the context until it's actually needed. That might >> not always be a reasonable plan -- in particular, it

Re: Error generating coverage report

2017-12-12 Thread Peter Eisentraut
On 12/12/17 15:39, David Steele wrote: > $ make -C test/build/src/bin/pg_basebackup coverage-html > make: Entering directory `/home/vagrant/test/build/src/bin/pg_basebackup' > /usr/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i -d . -d > /home/vagrant/src/src/bin/pg_basebackup -o

Re: [HACKERS] Custom compression methods

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 1:06 PM, Alexander Korotkov wrote: > OK, but NOTICE that presumably unexpected table rewrite takes place could be > still useful. I'm not going to complain too much about that, but I think that's mostly a failure of expectation rather than a

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Robert Haas
On Tue, Dec 12, 2017 at 10:02 AM, Geoff Winkless wrote: > There is also stackexchange question with 51000 views that asks how to > start the postgresql client. Should we rename psql to mysql to help > new users too? That's not a serious question, and I don't know what your

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Robert Haas
On Tue, Dec 12, 2017 at 11:58 AM, Everaldo Canuto wrote: > Sorry to be so sarcastic on last comment but, I am a little frustrated, it > is not like this patch is big and need a lots of review, it is already done > and it takes 15 minutes from me and it was the first

Re: Error generating coverage report

2017-12-12 Thread David Steele
Hi Peter, On 12/12/17 3:20 PM, Peter Eisentraut wrote: make: Entering directory `/home/vagrant/test/build/src/bin/pg_basebackup' /usr/bin/lcov --gcov-tool /usr/bin/gcov -q --no-external -c -i -d . -d /postgres/src/bin/pg_basebackup -o lcov_base.info geninfo: ERROR: no .gcno files found in

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Geoff Winkless
On 12 December 2017 at 18:32, Chapman Flack wrote: > It seems to me that this is very close to the Robert Haas suggestion > already discussed upthread, except that suggestion only prints the > hint message when you might need it, which I'd find less intrusive. Well "close"

Re: Learned Indexes in PostgreSQL?

2017-12-12 Thread Robert Haas
On Tue, Dec 12, 2017 at 6:38 AM, Stefan Keller wrote: > This is an awesome paper about a new index called "Learned Index". > it's aka dense hash structure derived ("learned") from actual data. > Looks very promising for certain properties [*]. > > Anyone already working on

Re: Backfill bgworker Extension?

2017-12-12 Thread Peter Eisentraut
On 12/12/17 13:03, Jeremy Finzel wrote: > To be clear, what I mean is batch updating a large set of data in small > pieces so as to avoid things like lock contention and replication lags.  > Sometimes these have a driving table that has the source data to update > in a destination table based on a

Re: Error generating coverage report

2017-12-12 Thread Peter Eisentraut
On 12/12/17 15:02, David Steele wrote: > I'm working on improving coverage and would like to generate some > reports (other than the text versions) to help me find uncovered code. > However, my source path and build path are not the same and I'm running > into problems. Vpath builds were one of

Re: [HACKERS] Custom compression methods

2017-12-12 Thread Oleg Bartunov
On Tue, Dec 12, 2017 at 6:07 PM, Alexander Korotkov wrote: > Hi! > > Let me add my two cents too. > > On Tue, Dec 12, 2017 at 2:41 PM, Ildus Kurbangaliev > wrote: >> >> On Mon, 11 Dec 2017 20:53:29 +0100 Tomas Vondra >>

Error generating coverage report

2017-12-12 Thread David Steele
I'm working on improving coverage and would like to generate some reports (other than the text versions) to help me find uncovered code. However, my source path and build path are not the same and I'm running into problems. This works fine and produces gcov output: $ make -C

Re: Rethinking MemoryContext creation

2017-12-12 Thread Andres Freund
On 2017-12-12 14:50:37 -0500, Robert Haas wrote: > On Tue, Dec 12, 2017 at 2:30 PM, Tom Lane wrote: > > 379 CurTransactionContext was never used > > 24 CurTransactionContext was used > > 66978 ExprContext was never used > > 17364 ExprContext was used > > 11139

Re: Rethinking MemoryContext creation

2017-12-12 Thread Robert Haas
On Tue, Dec 12, 2017 at 2:30 PM, Tom Lane wrote: > 379 CurTransactionContext was never used > 24 CurTransactionContext was used > 66978 ExprContext was never used > 17364 ExprContext was used > 11139 SPI Exec was never used >2421 SPI Exec was used > 38386

Re: Rethinking MemoryContext creation

2017-12-12 Thread Tom Lane
I wrote: > I've not done any benchmarking on this yet, just confirmed that it > compiles and passes check-world. So once I'd done some benchmarking, I was a bit disappointed: I could not measure any difference anymore in "pgbench -S", and poking into other scenarios found cases that were actually

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-12 Thread Andres Freund
On 2017-12-12 14:16:59 -0500, Robert Haas wrote: > On Tue, Dec 12, 2017 at 2:12 PM, Andres Freund wrote: > > I agree that it's more reliable - I hope there's no meaningful safety > > difference. I think you overestimate users a bit however - far from > > most of them are

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-12 Thread Robert Haas
On Tue, Dec 12, 2017 at 2:12 PM, Andres Freund wrote: > I agree that it's more reliable - I hope there's no meaningful safety > difference. I think you overestimate users a bit however - far from > most of them are going to be able to extract a very long log entry from > a

Re: ALTER TABLE ADD COLUMN fast default

2017-12-12 Thread Andrew Dunstan
On 12/06/2017 11:26 AM, Andrew Dunstan wrote: >> In general, you need to be thinking about this in terms of making sure >> that it's impossible to accidentally not update code that needs to be >> updated. Another example is that I noticed that some places the patch >> has

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-12 Thread Andres Freund
On 2017-12-12 14:04:55 -0500, Robert Haas wrote: > On Tue, Dec 12, 2017 at 1:50 PM, Andres Freund wrote: > > I do wonder if the right thing here wouldn't be to put the result into a > > dsm segment, and then return that to the UDF on the requesting > > side. Logging to the

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-12 Thread Robert Haas
On Tue, Dec 12, 2017 at 1:50 PM, Andres Freund wrote: > I do wonder if the right thing here wouldn't be to put the result into a > dsm segment, and then return that to the UDF on the requesting > side. Logging to the server log and then have the requestor dig that out >

Re: [HACKERS] parallel.c oblivion of worker-startup failures

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 10:41 PM, Amit Kapila wrote: > In particular, if the worker crashed (say somebody forcefully killed > the worker), then I think it will lead to a restart of the server. So > not sure, adding anything about that in the comment will make much >

Re: Using ProcSignal to get memory context stats from a running backend

2017-12-12 Thread Andres Freund
On 2017-12-12 14:25:48 +0800, Craig Ringer wrote: > If we were willing to wrap variadic calls in a callback and rely on > vfprintf, we could: I don't think there's problems with relying on variadic calls, we do that in plenty places. > and let the ProcSignal based caller pass an elog wrapper

Re: pgbench's expression parsing & negative numbers

2017-12-12 Thread Andres Freund
On 2017-12-12 07:21:21 +0100, Fabien COELHO wrote: > > Hello Andres, > > > working on overflow correctness in pg I noticed that pgbench isn't quite > > there. I assume it won't matter terribly often, but the way it parses > > integers makes it incorrect for, at least, the negativemost number. >

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Chapman Flack
On 12/12/2017 12:00 PM, Geoff Winkless wrote: > I quite like this idea, although I would rather a line printed after > each command is run: something like > > mydbname=# select ; > -- > (1 row) > > \h for help, \q for quit, \expert to turn off this message > osprey=# It seems to me that this

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Andres Freund
Hi, On 2017-12-12 11:01:04 -0500, Tom Lane wrote: > Robert Haas writes: > > Long story short, I don't think anyone cares about this enough to > > spend effort fixing it. I suspect the money data type has very few > > users. I'm unfortunately not so sure :(. > Somebody

Backfill bgworker Extension?

2017-12-12 Thread Jeremy Finzel
One of our challenges we have is that our engineers have written frameworks to backfill data in several different DSLs, and every time they adopt a new language, they maybe need to write another one. To be clear, what I mean is batch updating a large set of data in small pieces so as to avoid

Re: Learned Index

2017-12-12 Thread Oleg Ivanov
On 12/12/2017 12:16 PM, Laurenz Albe wrote: I have read into the paper. This may be interesting or not, but the paper is very vague about its concepts and algorithms, so it's hard to tell. I'd say that the paper does not meet publication standards. For example, they say that their results

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2017-12-12 Thread Oleg Ivanov
On 12/12/2017 04:33 PM, Oleg Bartunov wrote: On Mon, Dec 11, 2017 at 11:11 PM, Nikolay Samokhvalov wrote: Very interesting read: https://arxiv.org/abs/1712.01208 HN discussion: https://news.ycombinator.com/item?id=15894896 Some of the comments (from Twitter

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Geoff Winkless
On 12 December 2017 at 16:36, Arthur Nascimento wrote: > Nano keeps a cheatsheet of commands on the bottom; and it's regarded > as a fairly good editor for newcomers. Some shells have a right-hand > side status bar. I quite like this idea, although I would rather a line printed

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Everaldo Canuto
On Tue, Dec 12, 2017 at 2:39 PM, Everaldo Canuto wrote: > > As others have pointed out, there are _far_ more frustrating things >> (like the lack of SHOW CREATE [TABLE|etc]... support in the backend) >> for anyone coming from mysql that I would suggest should be

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Geoff Winkless
On 12 December 2017 at 16:39, Everaldo Canuto wrote: > On Tue, Dec 12, 2017 at 1:02 PM, Geoff Winkless wrote: >> There is also stackexchange question with 51000 views that asks how to >> start the postgresql client. Should we rename psql to mysql

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Everaldo Canuto
On Tue, Dec 12, 2017 at 1:02 PM, Geoff Winkless wrote: > > There is also stackexchange question with 51000 views that asks how to > start the postgresql client. Should we rename psql to mysql to help > new users too? > No, we shouldn't but it will help a lot is postgres

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Arthur Nascimento
On 12 December 2017 at 13:04, Jeremy Finzel wrote: > FWIW I am +1 in favor of not overcomplicating the special psql commands and > keep to the "\" standard. I find the general rule of \ starts a client command and anything else starts a server command to be very good to begin

Re: Boolean partitions syntax

2017-12-12 Thread Peter Eisentraut
On 12/12/17 04:12, Ashutosh Bapat wrote: > May be you should use opt_boolean_or_string instead of TRUE_P and > FALSE_P. It also supports ON and OFF, which will be bonus. But ON and OFF are not valid boolean literals in SQL. -- Peter Eisentraut http://www.2ndQuadrant.com/ PostgreSQL

Re: plpgsql test layout

2017-12-12 Thread Peter Eisentraut
On 12/11/17 19:29, Michael Paquier wrote: > If I read vcregress.pl correctly, it seems to me that you need to do > more with MSVC (see plcheck). The tests would kick if sql/ and > expected/ are found, and the test list is fetched by looking at > REGRESSION in the test files. However plpgsql code

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Tom Lane
Robert Haas writes: > Long story short, I don't think anyone cares about this enough to > spend effort fixing it. I suspect the money data type has very few > users. Somebody did contribute the effort not too long ago to install that overflow check into cash_in. So maybe

Re: Incorrect debug info printed in generate_partition_wise_join_paths

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 9:05 PM, Etsuro Fujita wrote: > (2017/12/11 17:24), Ashutosh Bapat wrote: >> Yes, that's the correct fix. We should be printing debug information >> about the child and not the parent. > > Thanks for reviewing! Committed. -- Robert Haas

Re: money type's overflow handling is woefully incomplete

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 5:50 PM, Andres Freund wrote: > This leads to fun like: > > postgres[2002][1]=# SELECT '92233720368547758.07'::money+'0.1'; > ┌─┐ > │ ?column? │ > ├─┤ > │

Re: [HACKERS] Moving relation extension locks out of heavyweight lock manager

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 4:10 PM, Masahiko Sawada wrote: > Thank you for updating the patch. Here is two minor comments. > > + * we acquire the same relation extension lock repeatedly. nLocks is 0 is > the > + * number of times we've acquired that lock; > > Should it be

portal pinning

2017-12-12 Thread Peter Eisentraut
While working on transaction control in procedures, I noticed some inconsistencies in how portal pinning is used. This mechanism was introduced in eb81b6509f4c9109ecf8839d8c482cc597270687 to prevent user code from closing cursors that PL/pgSQL has created internally, mainly for FOR loops.

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Avinash Kumar
It takes a few hours to learn PostgreSQL and 1 second to remember, how to quit from psql. Surprised to see the depth of discussion on this. It takes sometime for a person to explore or know about PostgreSQL and install it. Likewise, it takes him some more time to understand that there exists a

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread David G. Johnston
On Tue, Dec 12, 2017 at 8:04 AM, Jeremy Finzel wrote: > I don't think this is a significant stumbling block for users adopting > postgres. > ​The OP complained, and I tend to agree, that 90% of the world uses the command "exit" to end a program and that it would be nice if we

Re: [HACKERS] Custom compression methods

2017-12-12 Thread Alexander Korotkov
Hi! Let me add my two cents too. On Tue, Dec 12, 2017 at 2:41 PM, Ildus Kurbangaliev < i.kurbangal...@postgrespro.ru> wrote: > On Mon, 11 Dec 2017 20:53:29 +0100 Tomas Vondra < > tomas.von...@2ndquadrant.com> wrote: > > Replacing the algorithm used to compress all varlena values (in a way > >

Re: proposal: alternative psql commands quit and exit

2017-12-12 Thread Jeremy Finzel
Someone mentioned about needing to read the documentation of vim to learn how to exit the program. I personally think exactly the same applies here, and am a bit surprised at the depth of discussion over this. When I first was new to cli programs, the only "standard" way to exit a program I

Re: [HACKERS] Transactions involving multiple postgres foreign servers

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 5:20 AM, Masahiko Sawada wrote: >> The question I have is how would we deal with a foreign server that is >> not available for longer duration due to crash, longer network outage >> etc. Example is the foreign server crashed/got disconnected after >>

Re: Testing Extension Upgrade Paths

2017-12-12 Thread Jeremy Finzel
On Mon, Dec 11, 2017 at 7:55 PM, Craig Ringer wrote: > On 12 December 2017 at 07:25, Michael Paquier > wrote: > >> On Tue, Dec 12, 2017 at 2:55 AM, Jeremy Finzel wrote: >> > I understand how to setup a regression suite for a

Re: Out of date comment in cached_plan_cost

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 5:00 AM, David Rowley wrote: > On 11 December 2017 at 21:39, Ashutosh Bapat > wrote: >> I don't see much difference in the old and new wording. The word >> "generally" confuses more than clarifying the cases

Re: Inconsistency in plpgsql's error context reports

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 3:52 PM, Tom Lane wrote: > Here's a quick hack at that. I guess the main question that needs to be > asked is whether we're happy with plpgsql getting so much chattier > (as per all the regression test changes). > > If we're not, the alternative that

Re: [HACKERS] Proposal: Local indexes for partitioned table

2017-12-12 Thread Robert Haas
On Mon, Dec 11, 2017 at 10:04 PM, Alvaro Herrera wrote: > David Rowley wrote: > >> ATTACH/REPLACE sounds fine. My objection was more about the >> DETACH/ATTACH method to replace an index. > > So what happens if you do ALTER INDEX .. ATTACH and you already have > another

Re: ML-based indexing ("The Case for Learned Index Structures", a paper from Google)

2017-12-12 Thread Oleg Bartunov
On Mon, Dec 11, 2017 at 11:11 PM, Nikolay Samokhvalov wrote: > Very interesting read: https://arxiv.org/abs/1712.01208 > > HN discussion: https://news.ycombinator.com/item?id=15894896 > > Some of the comments (from Twitter >

Re: CUBE seems a bit confused about ORDER BY

2017-12-12 Thread Alexander Korotkov
On Tue, Dec 12, 2017 at 3:49 PM, Teodor Sigaev wrote: > Yes, the thing is that we change behavior of existing ~> operator. In >> general, this is not good idea because it could affect existing users whose >> already use this operator. Typically in such situation, we could

Re: CUBE seems a bit confused about ORDER BY

2017-12-12 Thread Teodor Sigaev
Yes, the thing is that we change behavior of existing ~> operator.  In general, this is not good idea because it could affect existing users whose already use this operator.  Typically in such situation, we could leave existing operator as is, and invent new operator with new behavior. 

Re: [HACKERS] logical decoding of two-phase transactions

2017-12-12 Thread Simon Riggs
On 12 December 2017 at 12:04, Nikhil Sontakke wrote: > This is an issue because of the way we are handling invalidations. We > don't allow ReorderBufferAddInvalidations() at COMMIT PREPARE time > since we assume that handling them at PREPARE time is enough. > Apparently,

Re: Fwd: [BUGS] pg_trgm word_similarity inconsistencies or bug

2017-12-12 Thread Teodor Sigaev
0001-pg-trgm-word-similarity-docs-improvement.patch – contains improvement to documentation of word_similarity() and related operators.  I decided to give formal definition first (what exactly it internally does), and then example and some more human-understandable description.  This patch also

Re: [HACKERS] Add support for tuple routing to foreign partitions

2017-12-12 Thread Etsuro Fujita
Hi Maksim, (2017/12/12 17:57), Maksim Milyutin wrote: Your patch already is not applied on master. Please rebase it. Done. Please find attached an updated version of the patch. Best regards, Etsuro Fujita *** a/contrib/file_fdw/input/file_fdw.source ---

Re: [HACKERS] logical decoding of two-phase transactions

2017-12-12 Thread Nikhil Sontakke
Hi, Thanks for the warning fix, I will also look at the cassert case soon. I have been adding more test cases to this patch. I added a TAP test which now allows us to do a concurrent ROLLBACK PREPARED when the walsender is in the midst of decoding this very prepared transaction. Have added a

Re: Parallel Index Scan vs BTP_DELETED and BTP_HALF_DEAD

2017-12-12 Thread Amit Kapila
On Tue, Dec 12, 2017 at 4:00 PM, Kuntal Ghosh wrote: > On Mon, Dec 11, 2017 at 2:26 PM, Thomas Munro > wrote: >> On Mon, Dec 11, 2017 at 8:14 PM, Amit Kapila wrote: >> >>> Thanks for looking into it. I will see

  1   2   >