[HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Amit Langote
Hi, In a custom pgbench script, it seems convenient to be able to split a really long query to span multiple lines using an escape character (bash-style). Attached adds that capability to read_line_from_file() in pgbench.c For example, BEGIN; \setrandom 1 1650 UPDATE table \ SET col2 =

Re: [HACKERS] IMPORT FOREIGN SCHEMA statement

2014-05-26 Thread Albe Laurenz
Ronan Dunklau wrote: Since my last proposal didn't get any strong rebuttal, please find attached a more complete version of the IMPORT FOREIGN SCHEMA statement. I tried to follow the SQL-MED specification as closely as possible. This adds discoverability to foreign servers. The structure

Re: [HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-26 Thread Michael Paquier
On Fri, May 23, 2014 at 10:43 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: x86_64-w64-mingw32-gcc -O2 -Wall -Wmissing-prototypes -Wpointer-arith -Wdeclaration-after-statement -Wendif-labels -Wmissing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Fujii Masao
On Mon, May 26, 2014 at 6:50 PM, Amit Langote amitlangot...@gmail.com wrote: Hi, In a custom pgbench script, it seems convenient to be able to split a really long query to span multiple lines using an escape character (bash-style). Attached adds that capability to read_line_from_file() in

Re: [HACKERS] Priority table or Cache table

2014-05-26 Thread Fujii Masao
On Sun, May 25, 2014 at 6:52 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 05/20/2014 01:46 PM, Fujii Masao wrote: On Mon, Mar 17, 2014 at 1:16 PM, Haribabu Kommi kommi.harib...@gmail.com wrote: ... I Implemented a proof of concept patch to see whether the buffer pool split can improve

[HACKERS] Re-create dependent views on ALTER TABLE ALTER COLUMN ... TYPE?

2014-05-26 Thread ash
Hi Hackers, This came up recently on general list (and I've just hit the same issue today): http://www.postgresql.org/message-id/cab7npqtlmmn1ltb5we0v0do57ip0u73ykwzbzytaxdf1caw...@mail.gmail.com Why couldn't postgres re-create the dependent views automatically? I mean it knows which views

Re: [HACKERS] Spreading full-page writes

2014-05-26 Thread Fujii Masao
On Mon, May 26, 2014 at 6:52 AM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Here's an idea I tried to explain to Andres and Simon at the pub last night, on how to reduce the spikes in the amount of WAL written at beginning of a checkpoint that full-page writes cause. I'm just writing

Re: [HACKERS] Re-create dependent views on ALTER TABLE ALTER COLUMN ... TYPE?

2014-05-26 Thread Stephen Frost
ash, * ash (a...@commandprompt.com) wrote: This came up recently on general list (and I've just hit the same issue today): http://www.postgresql.org/message-id/cab7npqtlmmn1ltb5we0v0do57ip0u73ykwzbzytaxdf1caw...@mail.gmail.com Why couldn't postgres re-create the dependent views

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Amit Langote
On Mon, May 26, 2014 at 10:59 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, May 26, 2014 at 6:50 PM, Amit Langote amitlangot...@gmail.com wrote: Thoughts? IMO it's better if we can write SQL in multiples line *without* a tailing escape character, like psql's input file. Yeah, that

[HACKERS] pg_llog not mentioned in Database File Layout

2014-05-26 Thread Amit Langote
Hi, Just noticed pg_llog is not mentioned in the Database File Layout section. Wonder if it's an oversight? -- Amit -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Alvaro Herrera
Amit Langote wrote: On Mon, May 26, 2014 at 10:59 PM, Fujii Masao masao.fu...@gmail.com wrote: On Mon, May 26, 2014 at 6:50 PM, Amit Langote amitlangot...@gmail.com wrote: Thoughts? IMO it's better if we can write SQL in multiples line *without* a tailing escape character, like

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Tom Lane
Amit Langote amitlangot...@gmail.com writes: In a custom pgbench script, it seems convenient to be able to split a really long query to span multiple lines using an escape character (bash-style). Attached adds that capability to read_line_from_file() in pgbench.c This seems pretty likely to

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Tom Lane
Alvaro Herrera alvhe...@2ndquadrant.com writes: Amit Langote wrote: On Mon, May 26, 2014 at 10:59 PM, Fujii Masao masao.fu...@gmail.com wrote: IMO it's better if we can write SQL in multiples line *without* a tailing escape character, like psql's input file. Yeah, that would be much cleaner.

Re: [HACKERS] Priority table or Cache table

2014-05-26 Thread Hannu Krosing
On 05/26/2014 04:16 PM, Fujii Masao wrote: On Sun, May 25, 2014 at 6:52 PM, Hannu Krosing ha...@2ndquadrant.com wrote: On 05/20/2014 01:46 PM, Fujii Masao wrote: On Mon, Mar 17, 2014 at 1:16 PM, Haribabu Kommi kommi.harib...@gmail.com wrote: ... I Implemented a proof of concept patch to see

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Amit Langote
On Tue, May 27, 2014 at 12:44 AM, Tom Lane t...@sss.pgh.pa.us wrote: Amit Langote amitlangot...@gmail.com writes: In a custom pgbench script, it seems convenient to be able to split a really long query to span multiple lines using an escape character (bash-style). Attached adds that capability

Re: [HACKERS] IMPORT FOREIGN SCHEMA statement

2014-05-26 Thread Tom Lane
Albe Laurenz laurenz.a...@wien.gv.at writes: In addition to data type mapping questions (which David already raised) I have one problem when I think of the Oracle FDW: Oracle follows the SQL standard in folding table names to upper case. So this would normally result in a lot of PostgreSQL

Re: [HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-26 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: On Fri, May 23, 2014 at 10:43 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: c:/mingw/msys/1.0/home/pgrunner/bf/root/HEAD/pgsql.5100/../pgsql/contrib/pg_stat_statements/pg_stat_statements.c: In function 'pgss_ProcessUtility':

Re: [HACKERS] Spreading full-page writes

2014-05-26 Thread Robert Haas
On May 25, 2014, at 5:52 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Here's how this works out during replay: a) You start WAL replay from the latest checkpoint's Redo-pointer. When you see a WAL record that's been marked with XLR_FPW_SKIPPED, don't replay that record at all.

Re: [HACKERS] Spreading full-page writes

2014-05-26 Thread Heikki Linnakangas
On 26 May 2014 20:16:33 EEST, Robert Haas robertmh...@gmail.com wrote: On May 25, 2014, at 5:52 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: Here's how this works out during replay: a) You start WAL replay from the latest checkpoint's Redo-pointer. When you see a WAL record that's

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Tom Lane
Matteo Beccati p...@beccati.com writes: I'm attaching v2 of the patch. Here's a list of changes from v1: * Restored --with-ossp-uuid. Configure tries ossp support first, then falls back to Linux and BSD variants * md5.o and sha1.o are linked only when not using the ossp library * fixed a bug

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Andres Freund
On 2014-05-26 13:25:49 -0400, Tom Lane wrote: Matteo Beccati p...@beccati.com writes: I'm attaching v2 of the patch. Here's a list of changes from v1: * Restored --with-ossp-uuid. Configure tries ossp support first, then falls back to Linux and BSD variants Imo should be the other way

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Jeff Janes
On Monday, May 26, 2014, Tom Lane t...@sss.pgh.pa.us wrote: Amit Langote amitlangot...@gmail.com javascript:; writes: In a custom pgbench script, it seems convenient to be able to split a really long query to span multiple lines using an escape character (bash-style). Attached adds that

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Matteo Beccati
On 26/05/2014 19:31, Andres Freund wrote: On 2014-05-26 13:25:49 -0400, Tom Lane wrote: Matteo Beccati p...@beccati.com writes: I'm attaching v2 of the patch. Here's a list of changes from v1: * Restored --with-ossp-uuid. Configure tries ossp support first, then falls back to Linux and BSD

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Andres Freund
On 2014-05-26 11:44:31 -0400, Tom Lane wrote: Amit Langote amitlangot...@gmail.com writes: In a custom pgbench script, it seems convenient to be able to split a really long query to span multiple lines using an escape character (bash-style). Attached adds that capability to

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Tom Lane
Matteo Beccati p...@beccati.com writes: On 26/05/2014 19:31, Andres Freund wrote: On 2014-05-26 13:25:49 -0400, Tom Lane wrote: Matteo Beccati p...@beccati.com writes: * Restored --with-ossp-uuid. Configure tries ossp support first, then falls back to Linux and BSD variants Imo should be

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Tom Lane
Jeff Janes jeff.ja...@gmail.com writes: On Monday, May 26, 2014, Tom Lane t...@sss.pgh.pa.us wrote: This seems pretty likely to break existing scripts that happen to contain backslashes. Is it really worth the compatibility risk? Do you mean due to the bugs you point out, or in general? Is

Re: [HACKERS] SQL access to database attributes

2014-05-26 Thread Vik Fearing
On 05/26/2014 07:10 AM, Stephen Frost wrote: * Tom Lane (t...@sss.pgh.pa.us) wrote: I don't really object to doing an unlocked check for another such database, but I'm not convinced that additional locking to try to prevent a race is worth its keep. +1 on the nannyism, and +1 to ignoring the

Re: [HACKERS] Spreading full-page writes

2014-05-26 Thread Greg Stark
On Mon, May 26, 2014 at 1:22 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: The second record is generated before the checkpoint is finished and the checkpoint record is written. So it will be there. (if you crash before the checkpoint is finished, the in-progress checkpoint is no

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Alvaro Herrera
Andres Freund wrote: I've more than once wished for the capability. +1 -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Re-create dependent views on ALTER TABLE ALTER COLUMN ... TYPE?

2014-05-26 Thread David Fetter
On Mon, May 26, 2014 at 06:25:09PM +0400, ash wrote: Hi Hackers, This came up recently on general list (and I've just hit the same issue today): http://www.postgresql.org/message-id/cab7npqtlmmn1ltb5we0v0do57ip0u73ykwzbzytaxdf1caw...@mail.gmail.com Why couldn't postgres re-create the

[HACKERS] Stating the significance of Lehman Yao in the nbtree README

2014-05-26 Thread Peter Geoghegan
While talking to various people during pgCon, I was reminded that the nbtree README does a poor job of explaining the actual practical advantages of LY from a high level. The move right trick is initially mentioned only as an adjunct to a discussion of the special-case handling of root page

Re: [HACKERS] Re-create dependent views on ALTER TABLE ALTER COLUMN ... TYPE?

2014-05-26 Thread ash
David Fetter da...@fetter.org writes: On Mon, May 26, 2014 at 06:25:09PM +0400, ash wrote: Hi Hackers, This came up recently on general list (and I've just hit the same issue today):

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Josh Berkus
On 05/26/2014 08:52 AM, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Amit Langote wrote: On Mon, May 26, 2014 at 10:59 PM, Fujii Masao masao.fu...@gmail.com wrote: IMO it's better if we can write SQL in multiples line *without* a tailing escape character, like psql's input

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Christoph Berg
Re: Tom Lane 2014-05-26 26629.1401119...@sss.pgh.pa.us Yeah, that would be much cleaner. But that would require duplicating the lexing stuff to determine where quotes are and where commands end. There are already some cases where pgbench itself is the bottleneck; adding a lexing step

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Christoph Berg
Re: Tom Lane 2014-05-25 12508.1401045...@sss.pgh.pa.us Matteo Beccati p...@beccati.com writes: here's the latest version of my uuid changes patch, according to proposal (2) from Tom in the thread about OSSP-UUID[1]. Hmm ... this is not actually what I had in mind. Unless I'm misreading

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Tom Lane
Christoph Berg c...@df7cb.de writes: Re: Tom Lane 2014-05-25 12508.1401045...@sss.pgh.pa.us Hmm ... this is not actually what I had in mind. Unless I'm misreading the patch, this nukes the uuid-ossp extension entirely in favor of a new extension uuid (providing the same SQL functions with a

Re: [HACKERS] Allowing line-continuation in pgbench custom scripts

2014-05-26 Thread Amit Langote
On Tue, May 27, 2014 at 6:19 AM, Josh Berkus j...@agliodbs.com wrote: On 05/26/2014 08:52 AM, Tom Lane wrote: Alvaro Herrera alvhe...@2ndquadrant.com writes: Amit Langote wrote: On Mon, May 26, 2014 at 10:59 PM, Fujii Masao masao.fu...@gmail.com wrote: IMO it's better if we can write SQL in

[HACKERS] Extending MSVC scripts to support --with-extra-version

2014-05-26 Thread Michael Paquier
Hi all, Please find attached a patch extending support of --with-extra-version in the MSVC scripts. This is something I have been using internally, and I believe that it is useful for Windows packagers. Similarly to the other options, a new field called extraver is added in config_default.pl and

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Tom Lane
I wrote: Matteo Beccati p...@beccati.com writes: On 26/05/2014 19:31, Andres Freund wrote: On 2014-05-26 13:25:49 -0400, Tom Lane wrote: Matteo Beccati p...@beccati.com writes: * Restored --with-ossp-uuid. Configure tries ossp support first, then falls back to Linux and BSD variants Imo

Re: [HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-26 Thread Michael Paquier
On Tue, May 27, 2014 at 1:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: The best alternative I can think of is to use strncmp() to check for whether the head of the string matches COPY , and then perform the integer conversion using strtoull() #ifdef HAVE_STRTOULL and strtoul() otherwise. What

Re: [HACKERS] Re: popen and pclose redefinitions causing many warning in Windows build

2014-05-26 Thread Tom Lane
Michael Paquier michael.paqu...@gmail.com writes: On Tue, May 27, 2014 at 1:39 AM, Tom Lane t...@sss.pgh.pa.us wrote: The best alternative I can think of is to use strncmp() to check for whether the head of the string matches COPY , and then perform the integer conversion using strtoull()

Re: [HACKERS] postgres_fdw and connection management

2014-05-26 Thread Shigeru Hanada
2014-05-24 0:09 GMT+09:00 Sandro Santilli s...@keybit.net: Indeed I tried DISCARD ALL in hope it would have helped, so I find good your idea of allowing extensions to register an hook there. Still, I'd like the FDW handler itself to possibly be configured to disable the pool completely as a

Re: [HACKERS] fix worker_spi to run as non-dynamic background worker

2014-05-26 Thread Robert Haas
On Fri, May 23, 2014 at 5:29 AM, Shigeru Hanada shigeru.han...@gmail.com wrote: I noticed that contrib/worker_spi can't run as non-dynamic background worker (IOW, load via shared_preload_libraries), because of uninitialized bgw_notify_pid. Attached patch fixes this issue. Please apply onto

Re: [HACKERS] fix worker_spi to run as non-dynamic background worker

2014-05-26 Thread Robert Haas
On Fri, May 23, 2014 at 10:38 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, May 23, 2014 at 6:29 PM, Shigeru Hanada shigeru.han...@gmail.com wrote: I noticed that contrib/worker_spi can't run as non-dynamic background worker (IOW, load via shared_preload_libraries), because of

Re: [HACKERS] Spreading full-page writes

2014-05-26 Thread Robert Haas
On Mon, May 26, 2014 at 1:22 PM, Heikki Linnakangas hlinnakan...@vmware.com wrote: I don't think we know that. The server might have crashed before that second record got generated. (This appears to be an unfixable flaw in this proposal.) The second record is generated before the checkpoint is

Re: [HACKERS] postgres_fdw and connection management

2014-05-26 Thread Fabrízio de Royes Mello
On Mon, May 26, 2014 at 11:47 PM, Shigeru Hanada shigeru.han...@gmail.com wrote: 2014-05-24 0:09 GMT+09:00 Sandro Santilli s...@keybit.net: Indeed I tried DISCARD ALL in hope it would have helped, so I find good your idea of allowing extensions to register an hook there. Still, I'd like

Re: [HACKERS] Compression of full-page-writes

2014-05-26 Thread Rahila Syed
Hello All, 0001-CompressBackupBlock_snappy_lz4_pglz extends patch on compression of full page writes to include LZ4 and Snappy . Changes include making compress_backup_block GUC from boolean to enum. Value of the GUC can be OFF, pglz, snappy or lz4 which can be used to turn off compression or

Re: [HACKERS] fix worker_spi to run as non-dynamic background worker

2014-05-26 Thread Michael Paquier
On Tue, May 27, 2014 at 12:09 PM, Robert Haas robertmh...@gmail.com wrote: On Fri, May 23, 2014 at 10:38 AM, Michael Paquier michael.paqu...@gmail.com wrote: On Fri, May 23, 2014 at 6:29 PM, Shigeru Hanada shigeru.han...@gmail.com wrote: I noticed that contrib/worker_spi can't run as

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread David E. Wheeler
On May 26, 2014, at 6:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: This means that if we want to give users control over which implementation gets selected, we actually need *three* configure switches. In the attached revision of Matteo's patch, I called them --with-ossp-uuid (the existing

Re: [HACKERS] Re-create dependent views on ALTER TABLE ALTER COLUMN ... TYPE?

2014-05-26 Thread David Fetter
On Tue, May 27, 2014 at 12:37:32AM +0400, ash wrote: David Fetter da...@fetter.org writes: On Mon, May 26, 2014 at 06:25:09PM +0400, ash wrote: Hi Hackers, This came up recently on general list (and I've just hit the same issue today):

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Tom Lane
David E. Wheeler da...@justatheory.com writes: On May 26, 2014, at 6:07 PM, Tom Lane t...@sss.pgh.pa.us wrote: This means that if we want to give users control over which implementation gets selected, we actually need *three* configure switches. In the attached revision of Matteo's patch, I

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread David E. Wheeler
On May 26, 2014, at 9:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: How about --with-unix-uuid? Or --with-ext2-uuid? Meh. Unix certainly subsumes BSD, so that doesn't seem like a very useful distinction. I guess we could use ext2 but that would just confuse most people. --with-uuid? Which

Re: [HACKERS] [PATCH] Replacement for OSSP-UUID for Linux and BSD

2014-05-26 Thread Tom Lane
David E. Wheeler da...@justatheory.com writes: On May 26, 2014, at 9:30 PM, Tom Lane t...@sss.pgh.pa.us wrote: How about --with-unix-uuid? Or --with-ext2-uuid? Meh. Unix certainly subsumes BSD, so that doesn't seem like a very useful distinction. I guess we could use ext2 but that would

Re: [HACKERS] postgresql.auto.conf read from wrong directory

2014-05-26 Thread Amit Kapila
On Sun, May 11, 2014 at 11:23 PM, Tom Lane t...@sss.pgh.pa.us wrote: I think it's clearly *necessary* to forbid setting data_directory in postgresql.auto.conf. The file is defined to be found in the data directory, so any such setting is circular logic by definition; no good can come of not