Re: [HACKERS] Doc patch to note which system catalogs have oids

2012-12-14 Thread Jeff Davis
On Tue, 2012-10-02 at 10:46 -0500, Karl O. Pinc wrote: > I am now submitting patches to the commitfest > for review. (I'm not sure how I missed this.) I prefer this version of the patch. I also attached an alternative version that may address Tom's concern by noting that the OIDs are hidden in th

Re: [HACKERS] gistchoose vs. bloat

2012-12-14 Thread Jeff Davis
On Fri, 2012-12-14 at 01:03 +0400, Alexander Korotkov wrote: > Hi! > > On Sat, Dec 8, 2012 at 7:05 PM, Andres Freund > wrote: > I notice there's no documentation about the new reloption at > all? > > > Thanks for notice! I've added small description to docs in the > attached pat

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2012-12-14 Thread Kyotaro HORIGUCHI
Hello, I took the perfomance figures for this patch. CentOS6.3/Core i7 wal_level = archive, checkpoint_segments = 30 / 5min A. Vanilla pgbench, postgres is HEAD B. Vanilla pgbench, postgres is with this patch (wal_update_changes_lz_v5) C. Modified pgbench(Long text), postgres is HEAD D. Modified

Re: [HACKERS] gistchoose vs. bloat

2012-12-14 Thread Alexander Korotkov
On Fri, Dec 14, 2012 at 12:46 PM, Jeff Davis wrote: > > Thanks for notice! I've added small description to docs in the > > attached patch. > > Here is an edited version of the documentation note. Please review to > see if you like my version. > Edited version looks good for me. > Also, I fixed

Re: [HACKERS] Identity projection

2012-12-14 Thread Heikki Linnakangas
On 12.11.2012 12:07, Kyotaro HORIGUCHI wrote: Hello, This is new version of identity projection patch. Reverted projectionInfo and ExecBuildProjectionInfo. Identity projection is recognized directly in ExecGroup, ExecResult, and ExecWindowAgg. nodeAgg is reverted because I couldn't make it sane.

Re: [HACKERS] logical decoding - GetOldestXmin

2012-12-14 Thread Andres Freund
On 2012-12-13 23:35:00 +, Simon Riggs wrote: > On 13 December 2012 22:37, Andres Freund wrote: > > On 2012-12-13 17:29:06 -0500, Robert Haas wrote: > >> On Thu, Dec 13, 2012 at 3:03 PM, Andres Freund > >> wrote: > >> > It moves a computation of the sort of: > >> > > >> > result -= vacuum_def

Re: [HACKERS] Logical decoding & exported base snapshot

2012-12-14 Thread Andres Freund
On 2012-12-13 21:40:43 +0100, Andres Freund wrote: > On 2012-12-13 11:02:06 -0500, Steve Singer wrote: > > On 12-12-12 06:20 AM, Andres Freund wrote: > > >>Possible solutions: > > >>1) INIT_LOGICAL_REPLICATION waits for an answer from the client that > > >>confirms that logical replication initiali

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2012-12-14 Thread Amit Kapila
On Friday, December 14, 2012 2:32 PM Kyotaro HORIGUCHI wrote: > Hello, I took the perfomance figures for this patch. > > CentOS6.3/Core i7 > wal_level = archive, checkpoint_segments = 30 / 5min > > A. Vanilla pgbench, postgres is HEAD > B. Vanilla pgbench, postgres is with this patch > (wal_updat

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-12-14 Thread Amit Kapila
On Thursday, December 13, 2012 8:02 PM Merlin Moncure wrote: > On Thu, Dec 13, 2012 at 7:06 AM, Hari Babu > wrote: > > Please find the review of the patch. > > > Thanks for detailed review! > > > Basic stuff: > > > > - Patch applies with offsets. > > - Compiles cleanly with no warn

Re: [HACKERS] pl/python custom datatype parsers

2012-12-14 Thread Hannu Krosing
Did any (committed?) code result from this thread ? On 11/10/2011 09:13 PM, Peter Eisentraut wrote: On tis, 2011-11-08 at 16:08 -0500, Andrew Dunstan wrote: On 03/01/2011 11:50 AM, Peter Eisentraut wrote: On fre, 2011-02-11 at 16:49 +0100, Jan Urbański wrote: I believe it's (b). But as we do

Re: [HACKERS] Multiple --table options for other commands

2012-12-14 Thread Karl O. Pinc
On 12/13/2012 11:02:56 PM, Josh Kupershmidt wrote: > On Thu, Dec 13, 2012 at 9:03 PM, Karl O. Pinc wrote: > > Sorry to be so persnickety, and unhelpful until now. > > It seemed like it should be doable, but something > > was going wrong between keyboard and chair. I guess > > I should be doing t

[HACKERS] Assert for frontend programs?

2012-12-14 Thread Andrew Dunstan
As I'm working through the parallel dump patch, I notice this in one of the header files: #ifdef USE_ASSERT_CHECKING #define Assert(condition) \ if (!(condition)) \ { \ write_msg(NULL, "Failed assertion in %s, line %d\n", \ __FILE__, __LINE__); \ abort()

Re: [HACKERS] Use gcc built-in atomic inc/dec in lock.c

2012-12-14 Thread Mikko Tiihonen
On 12/13/2012 12:19 AM, Peter Geoghegan wrote: On 12 December 2012 22:11, Mikko Tiihonen wrote: noticed a "XXX: It might be worth considering using an atomic fetch-and-add instruction here, on architectures where that is supported." in lock.c Here is my first try at using it. That's interest

Re: [HACKERS] Doc patch to note which system catalogs have oids

2012-12-14 Thread Karl O. Pinc
On 12/14/2012 02:04:45 AM, Jeff Davis wrote: > On Tue, 2012-10-02 at 10:46 -0500, Karl O. Pinc wrote: > > I am now submitting patches to the commitfest > > for review. (I'm not sure how I missed this.) > > I prefer this version of the patch. I also attached an alternative > version that may addre

Re: [HACKERS] Doc patch, index search_path where it's used to secure functions

2012-12-14 Thread Karl O. Pinc
On 12/13/2012 10:05:15 PM, Peter Eisentraut wrote: > The other configuration parameters are all indexed as "x_y_z > configuration parameter", so I've kept search_path aligned with that. > I > have applied your other changes, so I think it's good now. Let me > know > if you feel additional change

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Tom Lane
Andrew Dunstan writes: > As I'm working through the parallel dump patch, I notice this in one of > the header files: > #ifdef USE_ASSERT_CHECKING > #define Assert(condition) \ > if (!(condition)) \ > { \ > write_msg(NULL, "Failed assertion in %s, line %d\n", \ >

Re: [HACKERS] Use gcc built-in atomic inc/dec in lock.c

2012-12-14 Thread Merlin Moncure
On Fri, Dec 14, 2012 at 9:33 AM, Mikko Tiihonen wrote: > On 12/13/2012 12:19 AM, Peter Geoghegan wrote: >> >> On 12 December 2012 22:11, Mikko Tiihonen >> wrote: >>> >>> noticed a "XXX: It might be worth considering using an atomic >>> fetch-and-add >>> instruction here, on architectures where th

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Heikki Linnakangas
On 14.12.2012 17:54, Tom Lane wrote: Andrew Dunstan writes: As I'm working through the parallel dump patch, I notice this in one of the header files: #ifdef USE_ASSERT_CHECKING #define Assert(condition) \ if (!(condition)) \ { \ write_msg(NULL, "Failed assertion in %s,

Re: [HACKERS] Use gcc built-in atomic inc/dec in lock.c

2012-12-14 Thread Mikko Tiihonen
On 12/14/2012 05:55 PM, Merlin Moncure wrote: On Fri, Dec 14, 2012 at 9:33 AM, Mikko Tiihonen wrote: On 12/13/2012 12:19 AM, Peter Geoghegan wrote: On 12 December 2012 22:11, Mikko Tiihonen wrote: noticed a "XXX: It might be worth considering using an atomic fetch-and-add instruction here,

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Tom Lane
Heikki Linnakangas writes: > On 14.12.2012 17:54, Tom Lane wrote: >> BTW, I think psql already has a "psql_assert". > psql_assert looks like this: > #ifdef USE_ASSERT_CHECKING > #include > #define psql_assert(p) assert(p) > #else > ... > On my Linux system, a failure looks like this: > ~$ ./a

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Andrew Dunstan
On 12/14/2012 11:33 AM, Tom Lane wrote: Heikki Linnakangas writes: On 14.12.2012 17:54, Tom Lane wrote: BTW, I think psql already has a "psql_assert". psql_assert looks like this: #ifdef USE_ASSERT_CHECKING #include #define psql_assert(p) assert(p) #else ... On my Linux system, a failure lo

Re: [HACKERS] gistchoose vs. bloat

2012-12-14 Thread Heikki Linnakangas
One question: does the randomization ever help when building a new index? In the original test case, you repeatedly delete and insert tuples, and I can see how the index can get bloated in that case. But I don't see how bloat would occur when building the index from scratch. BTW, I don't much

Re: [HACKERS] gistchoose vs. bloat

2012-12-14 Thread Jeff Davis
On Fri, 2012-12-14 at 18:36 +0200, Heikki Linnakangas wrote: > One question: does the randomization ever help when building a new > index? In the original test case, you repeatedly delete and insert > tuples, and I can see how the index can get bloated in that case. But I > don't see how bloat w

Re: [HACKERS] Use of systable_beginscan_ordered in event trigger patch

2012-12-14 Thread Robert Haas
On Thu, Dec 13, 2012 at 6:25 PM, Tom Lane wrote: > Robert Haas writes: >> On Wed, Dec 12, 2012 at 3:51 PM, Dimitri Fontaine >> wrote: >>> Robert, does that ring a bell to you? I'm going to crawl the archives >>> tomorrow if not… > >> Yeah, I'm pretty sure you can't set event triggers of any kind

Re: [HACKERS] pg_upgrade problem with invalid indexes

2012-12-14 Thread Bruce Momjian
On Tue, Dec 11, 2012 at 03:12:37PM -0500, Bruce Momjian wrote: > On Fri, Dec 7, 2012 at 04:49:19PM -0500, Bruce Momjian wrote: > > On Fri, Dec 7, 2012 at 10:38:39PM +0100, Andres Freund wrote: > > > On 2012-12-07 16:30:36 -0500, Bruce Momjian wrote: > > > > On Fri, Dec 7, 2012 at 04:21:48PM -050

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-12-14 Thread Robert Haas
On Tue, Dec 11, 2012 at 12:59 AM, Jeff Davis wrote: > For every developer who says "wow, that mysql query just worked without > modification" there is another one who says "oh, I forgot to test with > option XYZ... postgres is too complex to support, I'm going to drop it > from the list of support

Re: [HACKERS] Use of systable_beginscan_ordered in event trigger patch

2012-12-14 Thread Tom Lane
Robert Haas writes: > On Thu, Dec 13, 2012 at 6:25 PM, Tom Lane wrote: >> I suspect there are still ways to shoot yourself in the foot with a >> broken event trigger, but it's not quite as trivial as I thought. > I'm smart enough not to doubt you, but I'd sure appreciate a hint as > to what you'

Re: [HACKERS] logical decoding - GetOldestXmin

2012-12-14 Thread Robert Haas
On Fri, Dec 14, 2012 at 6:46 AM, Andres Freund wrote: > Just moving that tidbit inside the lock seems to be the pragmatic > choice. GetOldestXmin is called > > * once per checkpoint > * one per index build > * once in analyze > * twice per vacuum > * once for HS feedback messages > > Nothing of th

Re: [HACKERS] Use of systable_beginscan_ordered in event trigger patch

2012-12-14 Thread Robert Haas
On Fri, Dec 14, 2012 at 2:00 PM, Tom Lane wrote: > Robert Haas writes: >> On Thu, Dec 13, 2012 at 6:25 PM, Tom Lane wrote: >>> I suspect there are still ways to shoot yourself in the foot with a >>> broken event trigger, but it's not quite as trivial as I thought. > >> I'm smart enough not to do

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-12-14 Thread Tom Lane
Robert Haas writes: > ... In more > than ten years of working with PostgreSQL, I've never encountered > where the restriction at issue here prevented a bug. It's only > annoyed me and broken my application code (when moving from PostgreSQL > 8.2 to PostgreSQL 8.3, never mind any other database!)

Re: [HACKERS] Enabling Checksums

2012-12-14 Thread Jeff Davis
On Wed, 2012-12-12 at 17:52 -0500, Greg Smith wrote: > I can take this on, as part of the QA around checksums working as > expected. The result would be a Python program; I don't have quite > enough time to write this in C or re-learn Perl to do it right now. But > this won't be a lot of code.

Re: [HACKERS] Enabling Checksums

2012-12-14 Thread Greg Smith
On 12/14/12 3:00 PM, Jeff Davis wrote: After some thought, I don't see much value in introducing multiple instances of corruption at a time. I would think that the smallest unit of corruption would be the hardest to detect, so by introducing many of them in one pass makes it easier to detect. T

Re: [HACKERS] MySQL search query is not executing in Postgres DB

2012-12-14 Thread Robert Haas
On Fri, Dec 14, 2012 at 2:11 PM, Tom Lane wrote: > Robert Haas writes: >> ... In more >> than ten years of working with PostgreSQL, I've never encountered >> where the restriction at issue here prevented a bug. It's only >> annoyed me and broken my application code (when moving from PostgreSQL

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Andrew Dunstan
On 12/14/2012 11:33 AM, Tom Lane wrote: Heikki Linnakangas writes: On 14.12.2012 17:54, Tom Lane wrote: BTW, I think psql already has a "psql_assert". psql_assert looks like this: #ifdef USE_ASSERT_CHECKING #include #define psql_assert(p) assert(p) #else ... On my Linux system, a failure lo

Re: [HACKERS] Use gcc built-in atomic inc/dec in lock.c

2012-12-14 Thread Robert Haas
On Wed, Dec 12, 2012 at 5:19 PM, Peter Geoghegan wrote: > On 12 December 2012 22:11, Mikko Tiihonen > wrote: >> noticed a "XXX: It might be worth considering using an atomic fetch-and-add >> instruction here, on architectures where that is supported." in lock.c >> >> Here is my first try at using

Re: [HACKERS] Re: bulk_multi_insert infinite loops with large rows and small fill factors

2012-12-14 Thread Robert Haas
On Wed, Dec 12, 2012 at 8:29 AM, David Gould wrote: > We lose noticable performance when we raise fill-factor above 10. Even 20 is > slower. Whoa. > During busy times these hosts sometimes fall into a stable state > with very high cpu use mostly in s_lock() and LWLockAcquire() and I think > PinB

Re: [HACKERS] PageIsAllVisible()'s trustworthiness in Hot Standby

2012-12-14 Thread Robert Haas
On Wed, Dec 12, 2012 at 12:31 PM, Pavan Deolasee wrote: > On Wed, Dec 12, 2012 at 1:35 AM, Robert Haas wrote: >> On Tue, Dec 4, 2012 at 12:10 PM, Pavan Deolasee >> wrote: >>> Hmm. Yeah, I do not have guts to prove that either. I'll probably write up a >>> comment for your consideration to explai

Re: [HACKERS] Use of systable_beginscan_ordered in event trigger patch

2012-12-14 Thread Dimitri Fontaine
Robert Haas writes: >> Robert, does that ring a bell to you? I'm going to crawl the archives >> tomorrow if not… > > Yeah, I'm pretty sure you can't set event triggers of any kind on > event triggers. You proposed to allow some stuff that would affect > "every command", but I yelled and screamed

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Tom Lane
Andrew Dunstan writes: > I noticed, BTW, that there are one or two places in backend code that > seem to call plain assert unconditionally, notably src/port/open.c, > src/backend/utils/adt/inet_net_pton.c and some contrib modules. That > seems undesirable. Should we need to look at turning thes

Re: [HACKERS] Use of systable_beginscan_ordered in event trigger patch

2012-12-14 Thread Robert Haas
On Fri, Dec 14, 2012 at 3:46 PM, Dimitri Fontaine wrote: > Robert Haas writes: >>> Robert, does that ring a bell to you? I'm going to crawl the archives >>> tomorrow if not… >> >> Yeah, I'm pretty sure you can't set event triggers of any kind on >> event triggers. You proposed to allow some stuf

Re: [HACKERS] Use gcc built-in atomic inc/dec in lock.c

2012-12-14 Thread Любен Каравелов
- Цитат от Mikko Tiihonen (mikko.tiiho...@nitorcreations.com), на 14.12.2012 в 17:33 - > On 12/13/2012 12:19 AM, Peter Geoghegan wrote: >> On 12 December 2012 22:11, Mikko Tiihonen >> wrote: >>> noticed a "XXX: It might be worth considering using an atomic fetch-and-add >>> instruction h

[HACKERS] Parser Cruft in gram.y

2012-12-14 Thread Dimitri Fontaine
Robert Haas writes: > ISTM that a PGC_SUSER GUC, as I proposed previously, would serve this > need adequately, without the cost of more cruft in gram.y. I can't help but think about the experiments you did some time ago about splitting the grammar into differents sub-grammars (for lack of a bette

[HACKERS] Adjusting elog behavior in bootstrap/standalone mode

2012-12-14 Thread Tom Lane
In <3688.1355509...@sss.pgh.pa.us> I complained > PS: one odd thing here is that the ereport(LOG) in > InstallXLogFileSegment isn't doing anything; otherwise we'd have gotten > a much more helpful error report about "could not link file". I don't > think we run the bootstrap mode with log_min_mess

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Peter Eisentraut
On 12/14/12 11:33 AM, Tom Lane wrote: > Works for me. So just rename that to Assert() and move it into > postgres-fe.h? Or just call assert() and don't invent our own layer? -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Andrew Dunstan
On 12/14/2012 04:23 PM, Peter Eisentraut wrote: On 12/14/12 11:33 AM, Tom Lane wrote: Works for me. So just rename that to Assert() and move it into postgres-fe.h? Or just call assert() and don't invent our own layer? Well, part of the point is that it lets you use Assert() in code that m

Re: [HACKERS] Parser Cruft in gram.y

2012-12-14 Thread Tom Lane
Dimitri Fontaine writes: > Now, what about splitting those grammars in order to freely add any new > production rules with or without new keywords for administrative > commands, without a blink of though about the main parser grammar. Let me explain to you why there will never be a situation wher

[HACKERS] Makefiles don't seem to remember to rebuild everything anymore

2012-12-14 Thread Tom Lane
In a fully-built source tree: $ cd pgsql/src/backend/parser $ make make: Nothing to be done for `all'. ... okay so far ... $ rm gram.o rm: remove regular file `gram.o'? y $ make make: Nothing to be done for `all'. WTF? If I also remove objfiles.txt then make wakes up and remembers it's supposed

Re: [HACKERS] Assert for frontend programs?

2012-12-14 Thread Tom Lane
Peter Eisentraut writes: > On 12/14/12 11:33 AM, Tom Lane wrote: >> Works for me. So just rename that to Assert() and move it into >> postgres-fe.h? > Or just call assert() and don't invent our own layer? Having the layer is a good thing, eg so that USE_ASSERT_CHECKING can control it, or so tha

Re: [HACKERS] Switching timeline over streaming replication

2012-12-14 Thread Josh Berkus
Heikki, Tested this on yesterday's snapshot. Worked great. Test: 4 Ubuntu 10.04 LTS Cloud Servers (GoGrid) Configuration: Compiled 9.3(12-12-12) with: pg_stat_statements, citext, ISN, btree_gist, pl/perl Setup Test: Master-Master Replicated to: master-replica using pg_basebackup -x.

Re: [HACKERS] Parser Cruft in gram.y

2012-12-14 Thread Dimitri Fontaine
Tom Lane writes: > Let me explain to you why there will never be a situation where we can > consider new keywords to be zero-cost. Thanks for taking the time to do this. > Splitting the grammar into multiple grammars is unlikely to do much to > improve this --- in fact, it could easily make matt

Re: [HACKERS] Re: bulk_multi_insert infinite loops with large rows and small fill factors

2012-12-14 Thread David Gould
On Fri, 14 Dec 2012 15:39:44 -0500 Robert Haas wrote: > On Wed, Dec 12, 2012 at 8:29 AM, David Gould wrote: > > We lose noticable performance when we raise fill-factor above 10. Even 20 is > > slower. > > Whoa. Any interest in a fill-factor patch to place exactly one row per page? That would be

Re: [HACKERS] Parser Cruft in gram.y

2012-12-14 Thread Kevin Grittner
Tom Lane wrote: > the parser tables are basically number-of-tokens wide by > number-of-states high. (In HEAD there are 433 tokens known to the > grammar, all but 30 of which are keywords, and 4367 states.) > > Splitting the grammar into multiple grammars is unlikely to do > much to improve this -

Re: [HACKERS] logical decoding - GetOldestXmin

2012-12-14 Thread Andres Freund
On 2012-12-14 14:01:30 -0500, Robert Haas wrote: > On Fri, Dec 14, 2012 at 6:46 AM, Andres Freund wrote: > > Just moving that tidbit inside the lock seems to be the pragmatic > > choice. GetOldestXmin is called > > > > * once per checkpoint > > * one per index build > > * once in analyze > > * twi

Re: [HACKERS] Switching timeline over streaming replication

2012-12-14 Thread Fujii Masao
On Sat, Dec 8, 2012 at 12:51 AM, Heikki Linnakangas wrote: > On 06.12.2012 15:39, Amit Kapila wrote: >> >> On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: >>> >>> On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: >>

Re: [HACKERS] Doc patch to note which system catalogs have oids

2012-12-14 Thread Peter Eisentraut
On Fri, 2012-12-14 at 00:04 -0800, Jeff Davis wrote: > On Tue, 2012-10-02 at 10:46 -0500, Karl O. Pinc wrote: > > I am now submitting patches to the commitfest > > for review. (I'm not sure how I missed this.) > > I prefer this version of the patch. I also attached an alternative > version that m

[HACKERS] is allow_nonpic_in_shlib still useful?

2012-12-14 Thread Peter Eisentraut
In the plperl and plpython makefiles we look for a shared library of libperl or libpython, and if it's not found, we check for allow_nonpic_in_shlib, and if that's yes, then we proceed anyway. Apparently, and IIRC, this was set up in a time when those shared libraries were not available through sta