Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-07 Thread Shigeru Hanada
Hi Tomas, I reviewed v6 patch, and found that several places need fix. Sorry for extra nitpickings. * I found another extra space after asterisk. + RelFileNode * nodes; * Curly bracket which starts code block should be at the head of next line. + /* extend

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Simon Riggs
On 7 January 2013 07:29, Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp wrote: Anyway, the compression speed in lz4 is very fast, so in my opinion, there is a room to improve the current implementation in pg_lzcompress. So why don't we use LZ4? -- Simon Riggs

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread John R Pierce
On 1/7/2013 1:10 AM, Simon Riggs wrote: On 7 January 2013 07:29, Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp wrote: Anyway, the compression speed in lz4 is very fast, so in my opinion, there is a room to improve the current implementation in pg_lzcompress. So why don't we use LZ4?

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Simon Riggs
On 7 January 2013 09:19, John R Pierce pie...@hogranch.com wrote: On 1/7/2013 1:10 AM, Simon Riggs wrote: On 7 January 2013 07:29, Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp wrote: Anyway, the compression speed in lz4 is very fast, so in my opinion, there is a room to improve the

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Andres Freund
On 2013-01-07 09:57:58 +, Simon Riggs wrote: On 7 January 2013 09:19, John R Pierce pie...@hogranch.com wrote: On 1/7/2013 1:10 AM, Simon Riggs wrote: On 7 January 2013 07:29, Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp wrote: Anyway, the compression speed in lz4 is very

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread John R Pierce
On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I seem to remember somebody even posting a patch to that effect? I agree that it's probably too late in the 9.3 cycle to start with this. so an

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Andres Freund
On 2013-01-07 02:21:26 -0800, John R Pierce wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I seem to remember somebody even posting a patch to that effect? I agree that it's probably too

[HACKERS] psql \l to accept patterns

2013-01-07 Thread Peter Eisentraut
Here is a patch for psql's \l command to accept patterns, like \d commands do. While at it, I also added an S option to show system objects and removed system objects from the default display. This might be a bit controversial, but it's how it was decided some time ago that the \d commands

[HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-07 Thread Amit Kapila
Observation is that whenever a checkpoint happens and the wal_level configured is hot_standby then one standby snapshot XLOG gets written with the information of running transaction. So if first time checkpoint happened at specified interval, it will create new XLOG in LogStandbySnapshot, due

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-07 Thread Simon Riggs
On 7 January 2013 12:39, Amit Kapila amit.kap...@huawei.com wrote: So We can modify to change this in function LogStandbySnapshot as below: running = GetRunningTransactionData(); if (running-xcnt 0) LogCurrentRunningXacts(running); So

[HACKERS] A very small typo in the comment

2013-01-07 Thread ??????
Hi, I am a student in China. I have read some source code of postgreSQL, though only a little. I think maybe there is typo in this file: src\backend\catalog\index.c. At line 649 in v9.2.0, line 653 in V9.2.2. The sentence is *indexInfo: same info executor uses to insert into the index, I

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-07 Thread Amit Kapila
On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January 2013 12:39, Amit Kapila amit.kap...@huawei.com wrote: So We can modify to change this in function LogStandbySnapshot as below: running = GetRunningTransactionData(); if (running-xcnt 0)

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Greg Stark
On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce pie...@hogranch.com wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I seem to remember somebody even posting a patch to that effect? I agree

Re: [HACKERS] Re: Proposal: Store timestamptz of database creation on pg_database

2013-01-07 Thread Stephen Frost
* Fabrízio de Royes Mello (fabriziome...@gmail.com) wrote: Understood... a COMMENT is a database object, then if we add a creation time column to pg_description/shdescription tables how we track his creation time? When it's NULL it doesn't exist, in this case, when it transistions from NULL,

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-07 Thread Simon Riggs
On 7 January 2013 13:33, Amit Kapila amit.kap...@huawei.com wrote: If we skip the WAL record in the way you suggest, we'd be unable to start quickly in some cases. If there are any operations happened which have generated WAL, then on next checkpoint interval the checkpoint operation should

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Simon Riggs
On 7 January 2013 13:36, Greg Stark st...@mit.edu wrote: On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce pie...@hogranch.com wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I seem to

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-07 Thread Andres Freund
On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January 2013 12:39, Amit Kapila amit.kap...@huawei.com wrote: So We can modify to change this in function LogStandbySnapshot as below: running =

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread k...@rice.edu
On Mon, Jan 07, 2013 at 09:10:31AM +, Simon Riggs wrote: On 7 January 2013 07:29, Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp wrote: Anyway, the compression speed in lz4 is very fast, so in my opinion, there is a room to improve the current implementation in pg_lzcompress. So

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread k...@rice.edu
On Mon, Jan 07, 2013 at 01:36:33PM +, Greg Stark wrote: On Mon, Jan 7, 2013 at 10:21 AM, John R Pierce pie...@hogranch.com wrote: On 1/7/2013 2:05 AM, Andres Freund wrote: I think there should be enough bits available in the toast pointer to indicate the type of compression. I seem to

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Andres Freund
Hi, It seems worth rereading the thread around http://archives.postgresql.org/message-id/CAAZKuFb59sABSa7gCG0vnVnGb-mJCUBBbrKiyPraNXHnis7KMw%40mail.gmail.com for people wanting to work on this. Greetings, Andres Freund -- Andres Freund http://www.2ndQuadrant.com/

Re: Review of pg_basebackup and pg_receivexlog to use non-blocking socket communication, was: Re: [HACKERS] Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2013-01-07 Thread Boszormenyi Zoltan
2013-01-04 13:43 keltezéssel, Hari Babu írta: On January 02, 2013 12:41 PM Hari Babu wrote: On January 01, 2013 10:19 PM Boszormenyi Zoltan wrote: I am reviewing your patch. • Is the patch in context diff format? Yes. Thanks for reviewing the patch. • Does it apply cleanly to the current

Re: [HACKERS] json api WIP patch

2013-01-07 Thread Merlin Moncure
On Fri, Jan 4, 2013 at 3:03 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I understand - but hstore isn't in core - so it should not be precedent regexp_split_to_table I am not native speaker, it sounds little bit strange - but maybe because I am not native speaker :) it's common usage:

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Tom Lane
Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp writes: The attached is a patch to improve compression speeds with loss of compression ratios in backend/utils/adt/pg_lzcompress.c. Why would that be a good tradeoff to make? Larger stored values require more I/O, which is likely to swamp any CPU

Re: [HACKERS] ALTER .. OWNER TO error mislabels schema as other object type

2013-01-07 Thread Robert Haas
On Wed, Jan 2, 2013 at 10:35 AM, Kohei KaiGai kai...@kaigai.gr.jp wrote: The attached patch fixes this trouble. Thanks. Committed. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list

Re: [HACKERS] [PERFORM] Slow query: bitmap scan troubles

2013-01-07 Thread Tom Lane
I wrote: Now, if we consider only CPU costs of index descent, we expect about log2(P) comparisons to be needed on each of the H upper pages to be descended through, that is we have total descent cost cpu_index_tuple_cost * H * log2(P) If we ignore the ceil() step as being a second-order

Re: [HACKERS] [PERFORM] Slow query: bitmap scan troubles

2013-01-07 Thread Simon Riggs
On 7 January 2013 17:35, Tom Lane t...@sss.pgh.pa.us wrote: That gives a formula of cpu_operator_cost * log2(N) + cpu_operator_cost * 50 * (H+2) This would lead to the behavior depicted in the attached plot, wherein I've modified the comparison lines (historical, 9.2, and HEAD

Re: [HACKERS] [PERFORM] Slow query: bitmap scan troubles

2013-01-07 Thread Tom Lane
Simon Riggs si...@2ndquadrant.com writes: On 7 January 2013 17:35, Tom Lane t...@sss.pgh.pa.us wrote: That gives a formula of cpu_operator_cost * log2(N) + cpu_operator_cost * 50 * (H+2) Again, this depends on N and H, so thats good. I think my retinas detached while reading your

Re: [HACKERS] [PERFORM] Slow query: bitmap scan troubles

2013-01-07 Thread Claudio Freire
On Mon, Jan 7, 2013 at 3:27 PM, Tom Lane t...@sss.pgh.pa.us wrote: One issue that needs some thought is that the argument for this formula is based entirely on thinking about b-trees. I think it's probably reasonable to apply it to gist, gin, and sp-gist as well, assuming we can get some

Re: [HACKERS] recent ALTER whatever .. SET SCHEMA refactoring

2013-01-07 Thread Alvaro Herrera
Kohei KaiGai escribió: Function and collation are candidates of this special case handling; here are just two kinds of object. Another idea is to add a function-pointer as argument of AlterNamespace_internal for (upcoming) object classes that takes special handling for detection of name

Re: [HACKERS] psql \l to accept patterns

2013-01-07 Thread Fabrízio de Royes Mello
On Mon, Jan 7, 2013 at 10:14 AM, Peter Eisentraut pete...@gmx.net wrote: Here is a patch for psql's \l command to accept patterns, like \d commands do. While at it, I also added an S option to show system objects and removed system objects from the default display. This might be a bit

Re: [HACKERS] psql \l to accept patterns

2013-01-07 Thread Alvaro Herrera
Peter Eisentraut wrote: Here is a patch for psql's \l command to accept patterns, like \d commands do. While at it, I also added an S option to show system objects and removed system objects from the default display. This might be a bit controversial, but it's how it was decided some time

Re: [HACKERS] json api WIP patch

2013-01-07 Thread Pavel Stehule
2013/1/7 Merlin Moncure mmonc...@gmail.com: On Fri, Jan 4, 2013 at 3:03 PM, Pavel Stehule pavel.steh...@gmail.com wrote: I understand - but hstore isn't in core - so it should not be precedent regexp_split_to_table I am not native speaker, it sounds little bit strange - but maybe because I

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Merlin Moncure
On Mon, Jan 7, 2013 at 10:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp writes: The attached is a patch to improve compression speeds with loss of compression ratios in backend/utils/adt/pg_lzcompress.c. Why would that be a good tradeoff to make?

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Tom Lane
Merlin Moncure mmonc...@gmail.com writes: On Mon, Jan 7, 2013 at 10:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp writes: The attached is a patch to improve compression speeds with loss of compression ratios in backend/utils/adt/pg_lzcompress.c.

Re: [HACKERS] ALTER command reworks

2013-01-07 Thread Alvaro Herrera
Kohei KaiGai escribió: The attached patch is a revised version. It follows the manner in ExecAlterObjectSchemaStmt(); that tries to check name duplication for object classes that support catcache with name-key. Elsewhere, it calls individual routines for each object class to solve object

Re: [HACKERS] recent ALTER whatever .. SET SCHEMA refactoring

2013-01-07 Thread Robert Haas
On Mon, Jan 7, 2013 at 2:14 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Kohei KaiGai escribió: Function and collation are candidates of this special case handling; here are just two kinds of object. Another idea is to add a function-pointer as argument of AlterNamespace_internal for

Re: [HACKERS] ALTER command reworks

2013-01-07 Thread Robert Haas
On Mon, Jan 7, 2013 at 3:43 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Kohei KaiGai escribió: The attached patch is a revised version. It follows the manner in ExecAlterObjectSchemaStmt(); that tries to check name duplication for object classes that support catcache with name-key.

Re: [HACKERS] psql \l to accept patterns

2013-01-07 Thread Robert Haas
On Mon, Jan 7, 2013 at 7:14 AM, Peter Eisentraut pete...@gmx.net wrote: Here is a patch for psql's \l command to accept patterns, like \d commands do. While at it, I also added an S option to show system objects and removed system objects from the default display. This might be a bit

Re: [HACKERS] ALTER command reworks

2013-01-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jan 7, 2013 at 3:43 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I checked this patch. It needed a rebase for the changes to return OIDs. Attached patch applies to current HEAD. In general this looks good, with one exception: it's

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Robert Haas
On Mon, Jan 7, 2013 at 11:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Why would that be a good tradeoff to make? Larger stored values require more I/O, which is likely to swamp any CPU savings in the compression step. Not to mention that a value once written may be read many times, so the

Re: [HACKERS] ALTER command reworks

2013-01-07 Thread Kohei KaiGai
2013/1/7 Tom Lane t...@sss.pgh.pa.us: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 7, 2013 at 3:43 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: I checked this patch. It needed a rebase for the changes to return OIDs. Attached patch applies to current HEAD. In general this

Re: [HACKERS] psql \l to accept patterns

2013-01-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jan 7, 2013 at 7:14 AM, Peter Eisentraut pete...@gmx.net wrote: Here is a patch for psql's \l command to accept patterns, like \d commands do. While at it, I also added an S option to show system objects and removed system objects from the

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Tom Lane
Robert Haas robertmh...@gmail.com writes: On Mon, Jan 7, 2013 at 11:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Why would that be a good tradeoff to make? Larger stored values require more I/O, which is likely to swamp any CPU savings in the compression step. Not to mention that a value once

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Merlin Moncure
On Mon, Jan 7, 2013 at 2:41 PM, Tom Lane t...@sss.pgh.pa.us wrote: Merlin Moncure mmonc...@gmail.com writes: On Mon, Jan 7, 2013 at 10:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Takeshi Yamamuro yamamuro.take...@lab.ntt.co.jp writes: The attached is a patch to improve compression speeds with

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Andrew Dunstan
On 01/07/2013 04:19 PM, Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 7, 2013 at 11:16 AM, Tom Lane t...@sss.pgh.pa.us wrote: Why would that be a good tradeoff to make? Larger stored values require more I/O, which is likely to swamp any CPU savings in the compression

Re: [HACKERS] ALTER command reworks

2013-01-07 Thread Alvaro Herrera
Tom Lane escribió: Robert Haas robertmh...@gmail.com writes: On Mon, Jan 7, 2013 at 3:43 PM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: If this is considered a problem, I think the way to fix it is to have a getObjectDescriptionOids() variant that quotes the object name in the

Re: [HACKERS] Improve compression speeds in pg_lzcompress.c

2013-01-07 Thread Robert Haas
On Mon, Jan 7, 2013 at 4:19 PM, Tom Lane t...@sss.pgh.pa.us wrote: Hm ... one of us is reading those results backwards, then. *looks* It's me. Sorry for the noise. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-07 Thread Tomas Vondra
On 7.1.2013 10:07, Shigeru Hanada wrote: Hi Tomas, I reviewed v6 patch, and found that several places need fix. Sorry for extra nitpickings. * I found another extra space after asterisk. + RelFileNode * nodes; Thanks, fixed. * Curly bracket which starts code block should be at

Re: [HACKERS] psql \l to accept patterns

2013-01-07 Thread Peter Eisentraut
On 1/7/13 3:53 PM, Robert Haas wrote: On Mon, Jan 7, 2013 at 7:14 AM, Peter Eisentraut pete...@gmx.net wrote: Here is a patch for psql's \l command to accept patterns, like \d commands do. While at it, I also added an S option to show system objects and removed system objects from the

Re: [HACKERS] psql \l to accept patterns

2013-01-07 Thread Robert Haas
On Mon, Jan 7, 2013 at 5:14 PM, Peter Eisentraut pete...@gmx.net wrote: We removed showing system functions and operators from \df and \do without S. Those are needed all the time. This was controversial at the time, but it's the way it is now. The definition of S, I suppose, is more like

Re: [HACKERS] in-catalog Extension Scripts and Control parameters (templates?)

2013-01-07 Thread Dimitri Fontaine
Hi, Please find attached a preliminary patch following the TEMPLATE ideas, and thanks in particular to Tom and Heikki for a practical design about how to solve that problem! Tom Lane t...@sss.pgh.pa.us writes: - Extension Scripts are now stored in the catalogs, right? Only for these

Re: [HACKERS] A very small typo in the comment

2013-01-07 Thread Bruce Momjian
On Sat, Jan 5, 2013 at 12:13:27AM +0800, 孟庆钟 wrote: Hi, I am a student in China. I have read some source code of postgreSQL, though only a little. I think maybe there is typo in this file: src\backend\ catalog\index.c. At line 649 in v9.2.0, line 653 in V9.2.2. The sentence is

Re: [HACKERS] A very small typo in the comment

2013-01-07 Thread Bruce Momjian
On Mon, Jan 7, 2013 at 07:46:46PM -0500, Bruce Momjian wrote: On Sat, Jan 5, 2013 at 12:13:27AM +0800, 孟庆钟 wrote: Hi, I am a student in China. I have read some source code of postgreSQL, though only a little. I think maybe there is typo in this file: src\backend\ catalog\index.c.

Re: [HACKERS] PATCH: optimized DROP of multiple tables within a transaction

2013-01-07 Thread Shigeru Hanada
Hi Tomas, On Tue, Jan 8, 2013 at 7:08 AM, Tomas Vondra t...@fuzzy.cz wrote: * I found another extra space after asterisk. + RelFileNode * nodes; Thanks, fixed. check * Curly bracket which starts code block should be at the head of next line. + /* extend

[HACKERS] lazy_vacuum_heap()'s removal of HEAPTUPLE_DEAD tuples

2013-01-07 Thread Noah Misch
Per this comment in lazy_scan_heap(), almost all tuple removal these days happens in heap_page_prune(): case HEAPTUPLE_DEAD: /* * Ordinarily, DEAD tuples would have been removed by

[HACKERS] PL/Python result object str handler

2013-01-07 Thread Peter Eisentraut
For debugging PL/Python functions, I'm often tempted to write something like rv = plpy.execute(...) plpy.info(rv) which prints something unhelpful like PLyResult object at 0xb461d8d8 By implementing a str handler for the result object, it now prints something like PLyResult status=5 nrows=2

Re: [HACKERS] bad examples in pg_dump README

2013-01-07 Thread Peter Eisentraut
On Sat, 2013-01-05 at 15:34 +0100, Magnus Hagander wrote: On Fri, Jan 4, 2013 at 3:36 AM, Josh Kupershmidt schmi...@gmail.com wrote: I propose slimming down the pg_dump README, keeping intact the introductory notes and details of the tar format. Do we need to keep it at all, really?

Re: [HACKERS] [PATCH] Patch to fix missing libecpg_compat.lib and libpgtypes.lib.

2013-01-07 Thread Peter Eisentraut
On Mon, 2012-11-19 at 14:03 +0800, JiangGuiqing wrote: hi I install postgresql-9.1.5 from source code on windows successfully. But there is not libecpg_compat.lib and libpgtypes.lib in the installed lib directory . If someone used functions of pgtypes or ecpg_compat library in ecpg

[HACKERS] pg_upgrade with parallel tablespace copying

2013-01-07 Thread Bruce Momjian
Pg_upgrade by default (without --link) copies heap/index files from the old to new cluster. This patch implements parallel heap/index file copying in pg_upgrade using the --jobs option. It uses the same infrastructure used for pg_upgrade parallel dump/restore. Here are the performance results:

Re: [HACKERS] Set visibility map bit after HOT prune

2013-01-07 Thread Pavan Deolasee
Sorry for a long pause on this thread. A new arrival at home kept me occupied all the time. This thread saw a lot of ideas and suggestions from different people. I don't think we had an agreement one way or the other on any of them, but let me summarize the discussion for archival and taking

Re: [HACKERS] json api WIP patch

2013-01-07 Thread james
The processing functions have been extended to provide populate_record() and populate_recordset() functions.The latter in particular could be useful in decomposing a piece of json representing an array of flat objects (a fairly common pattern) into a set of Postgres records in a single pass.