[HACKERS] Regarding Checkpoint Redo Record

2012-01-04 Thread Amit Kapila
Why PostgreSQL needs to write WAL record for Checkpoint when it maintains same information in pg_control file? This may be required in case we need information about more than one checkpoint as pg_control can hold information of only recent checkpoint. But I could not think of a case where more

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-07 Thread Amit Kapila
I think it is a good idea, and can help double-writes perform better in the case of lots of backend evictions. I don't understand this point, because from the data in your mail, it appears that when shared buffers are less means when more evictions can happen, the performance is less.

Re: [HACKERS] double writes using double-write buffer approach [WIP]

2012-02-13 Thread Amit Kapila
has to never do I/O it can show performance improvement as well as compare to full page writes. -Original Message- From: Dan Scales [mailto:sca...@vmware.com] Sent: Thursday, February 09, 2012 5:30 AM To: Amit Kapila Cc: PG Hackers Subject: Re: [HACKERS] double writes using double-write

Re: [HACKERS] Checkpoint sync pause

2012-02-13 Thread Amit Kapila
Without sorted checkpoints (or some other fancier method) you have to write out the entire pool before you can do any fsyncs. Or you have to do multiple fsyncs of the same file, with at least one occurring after the entire pool was written. With a sorted checkpoint, you can start issuing

Re: [HACKERS] client performance v.s. server statistics

2012-02-15 Thread Amit Kapila
So, is it client interface (ODBC, libpq) 's cost mainly due to TCP? The difference as compare to your embedded DB you are seeing is mainly seems to be due to TCP. One optimization you can use is to use Unix-domain socket mode of PostgreSQL. You can refer unix_socket_directory parameter in

Re: [HACKERS] client performance v.s. server statistics

2012-02-15 Thread Amit Kapila
, February 15, 2012 12:32 PM To: Amit Kapila Cc: pgsql-hackers@postgresql.org Subject: Re: [HACKERS] client performance v.s. server statistics Hi, I have tried unix domain socket and the performance is similar with TCP socket. It is MIPS architecture so memory copy to/from kernel can occupy much

Re: Scaling XLog insertion (was Re: [HACKERS] Moving more work outside WALInsertLock)

2012-02-19 Thread Amit Kapila
I was trying to understand this patch and had few doubts: 1. In PerformXLogInsert(), why there is need to check freespace when already during ReserveXLogInsertLocation(), the space is reserved. Is it possible that the record size is more than actually calculted in

Re: [HACKERS] Parameterized-path cost comparisons need some work

2012-02-29 Thread Amit Kapila
The most obvious thing to do about this is to consider that one path can dominate another on cost only if it is both cheaper *and* produces no more rows. But I'm concerned about the cost of inserting yet another test condition into add_path, which is slow enough already. Has anyone got an idea

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-14 Thread Amit kapila
On Tuesday, November 13, 2012 9:29 AM Amit kapila wrote: On Monday, November 12, 2012 12:07 PM Greg Smith wrote: On 11/9/12 11:59 PM, Amit kapila wrote: 1) Change to add scanning a .conf directory in the default configuration using include-dir. This is a quick fix. I predict most

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-14 Thread Amit Kapila
On Wednesday, November 14, 2012 9:12 PM Fujii Masao wrote: On Wed, Nov 14, 2012 at 5:35 PM, Amit Kapila amit.kap...@huawei.com wrote: On Tuesday, November 13, 2012 10:17 PM Fujii Masao wrote: On Tue, Nov 13, 2012 at 1:23 PM, Amit kapila amit.kap...@huawei.com wrote: On Monday, November

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-14 Thread Amit Kapila
On Wednesday, November 14, 2012 10:19 PM Fujii Masao wrote: On Thu, Nov 15, 2012 at 12:55 AM, Amit Kapila amit.kap...@huawei.com wrote: Now user can use this utility to decide if new-standby has max LSN greater And that situation can occur when new-standby has startpoint LSN greater

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 2:02 AM Atri Sharma wrote: On Thu, Nov 15, 2012 at 12:42 AM, Atri Sharma atri.j...@gmail.com wrote: On Wed, Nov 7, 2012 at 9:47 PM, Merlin Moncure mmonc...@gmail.com wrote: On Wed, Nov 7, 2012 at 6:01 AM, Amit Kapila amit.kap...@huawei.com wrote: Following

Re: [HACKERS] Switching timeline over streaming replication

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 6:05 PM Heikki Linnakangas wrote: On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote

[HACKERS] Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-15 Thread Amit kapila
On Monday, November 12, 2012 8:23 PM Fujii Masao wrote: On Fri, Nov 9, 2012 at 3:03 PM, Amit Kapila amit.kap...@huawei.com wrote: On Thursday, November 08, 2012 10:42 PM Fujii Masao wrote: On Thu, Nov 8, 2012 at 5:53 PM, Amit Kapila amit.kap...@huawei.com wrote: On Thursday, November 08, 2012

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-15 Thread Amit kapila
On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila amit.kap...@huawei.com wrote: Uh, no, I don't think that's a good idea. IMHO, what we should do is: 1. Read postgresql.conf.auto and remember all the settings we saw. If we see

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 9:27 PM Merlin Moncure wrote: On Thu, Nov 15, 2012 at 4:39 AM, Amit Kapila amit.kap...@huawei.com wrote: In each visibility function (except HeapTupleSatisfiesVacuum() ), an addition check has been added to check if the commit status of Xmin or Xmax

Re: [HACKERS] autovacuum truncate exclusive lock round two

2012-11-15 Thread Amit Kapila
to complete the truncate. So I think it's better to complete first time only, but may be using some heuristic time for wait and retry rather than with configuration variables. With Regards, Amit Kapila -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-15 Thread Amit Kapila
On Thursday, November 15, 2012 11:28 PM Cédric Villemain wrote: Le jeudi 15 novembre 2012 15:48:14, Amit kapila a écrit : On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila amit.kap...@huawei.com wrote: Uh, no, I don't think that's

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-16 Thread Amit Kapila
From: Cédric Villemain [mailto:ced...@2ndquadrant.com] Sent: Friday, November 16, 2012 1:55 PM To: pgsql-hackers@postgresql.org Cc: Amit Kapila; 'Robert Haas'; 'Greg Smith'; 'Josh Berkus'; 'Tom Lane'; 'Magnus Hagander'; 'Christopher Browne' Subject: Re: [HACKERS] Proposal for Allow

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-11-16 Thread Amit Kapila
On Thursday, November 15, 2012 10:19 PM Merlin Moncure wrote: On Thu, Nov 15, 2012 at 10:25 AM, Amit Kapila amit.kap...@huawei.com wrote: Sure, although in scans we are using ring buffer as well so in practical sense the results are pretty close. b. Considering sometimes people want

Re: [HACKERS] [PATCH] Patch to compute Max LSN of Data Pages

2012-11-16 Thread Amit kapila
On Thursday, November 15, 2012 7:30 PM Robert Haas wrote: On Thu, Nov 15, 2012 at 12:08 AM, Amit Kapila amit.kap...@huawei.com wrote: Okay. So as Robert and Alvaro suggested to have it separate utility rather than having options in pg_resetxlog to print MAX LSN seems to be quite appropriate

Re: [HACKERS] Re: [BUGS] BUG #7534: walreceiver takes long time to detect n/w breakdown

2012-11-16 Thread Amit Kapila
On Thursday, November 15, 2012 7:29 PM Amit kapila wrote: On Monday, November 12, 2012 8:23 PM Fujii Masao wrote: On Fri, Nov 9, 2012 at 3:03 PM, Amit Kapila amit.kap...@huawei.com wrote: On Thursday, November 08, 2012 10:42 PM Fujii Masao wrote: On Thu, Nov 8, 2012 at 5:53 PM, Amit Kapila

Re: [HACKERS] Switching timeline over streaming replication

2012-11-16 Thread Amit Kapila
On Thursday, November 15, 2012 6:05 PM Heikki Linnakangas wrote: On 15.11.2012 12:44, Heikki Linnakangas wrote: Here's an updated version of this patch, rebased with master, including the recent replication timeout changes, and some other cleanup. On 12.10.2012 09:34, Amit Kapila wrote

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-16 Thread Amit Kapila
On Thursday, November 15, 2012 8:18 PM Amit kapila wrote: On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila amit.kap...@huawei.com wrote: Uh, no, I don't think that's a good idea. IMHO, what we should do is: 1. Read

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-17 Thread Amit Kapila
On Friday, November 16, 2012 7:52 PM Cédric Villemain wrote: Le vendredi 16 novembre 2012 15:08:30, Amit Kapila a écrit : On Thursday, November 15, 2012 8:18 PM Amit kapila wrote: On Wednesday, November 14, 2012 12:24 AM Robert Haas wrote: On Mon, Nov 12, 2012 at 10:59 PM, Amit kapila

Re: [HACKERS] another idea for changing global configuration settings from SQL

2012-11-17 Thread Amit Kapila
how and when the value will be used. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-18 Thread Amit Kapila
On Sunday, November 18, 2012 3:08 AM Fujii Masao wrote: On Sat, Nov 17, 2012 at 10:25 PM, Amit Kapila amit.kap...@huawei.com wrote: 1. have a system table pg_global_system_settings(key,value) Do we really need to store the settings in a system table? Since WAL would be generated when

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-18 Thread Amit Kapila
may not be required to work in transaction blocks as it will change in config file which can take effect only on re-start or sighup. I believe some more thoughts and suggestions are required to conclude. Thoughts/Suggestions/Comments? With Regards, Amit Kapila. -- Sent via pgsql

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-18 Thread Amit Kapila
it in sync (just use it to hide the complexity of locks). Anyway that was just comments. Thanks. You comments are thought provoking. I was able to proceed for table related approach based on your suggestions. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-19 Thread Amit Kapila
On Monday, November 19, 2012 7:53 PM Dimitri Fontaine wrote: Amit Kapila amit.kap...@huawei.com writes: We have discussion about below 3 different syntaxes for this command 1. ALTER SYSTEM 2. SET PERSISENT 3. pg_change_conf() I think to conclude, we need to evaluate which syntax has

Re: [HACKERS] [WIP PATCH] for Performance Improvement in Buffer Management

2012-11-19 Thread Amit kapila
On Monday, November 19, 2012 6:05 AM Jeff Janes wrote: On Mon, Oct 22, 2012 at 10:51 AM, Amit kapila amit.kap...@huawei.com wrote: Today again I have again collected the data for configuration Shared_buffers = 7G along with vmstat. The data and vmstat information (bi) are attached

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-19 Thread Amit Kapila
On Monday, November 19, 2012 8:36 PM Alvaro Herrera wrote: Amit Kapila escribió: The only point I can see against SET PERSISTENT is that other variants of SET command can be used in transaction blocks means for them ROLLBACK TO SAVEPOINT functionality works, but for SET PERSISTENT

Re: [HACKERS] Switching timeline over streaming replication

2012-11-20 Thread Amit Kapila
from the standby server, if the standby server got promoted as master the connection between pg_receivexlog and standby server is broken, because of this reason the current xlog file is not renamed as actual file. Whether such a scenario needs any handling? With Regards, Amit Kapila. -- Sent

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-20 Thread Amit Kapila
On Monday, November 19, 2012 9:07 PM Amit Kapila wrote: On Monday, November 19, 2012 8:36 PM Alvaro Herrera wrote: Amit Kapila escribió: The only point I can see against SET PERSISTENT is that other variants of SET command can be used in transaction blocks means for them ROLLBACK

Re: [HACKERS] StrategyGetBuffer questions

2012-11-20 Thread Amit Kapila
. In that case, why don't we work towards reducing it? Is the generic use case a problem or will it effect any generic scenario in negative way? With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] StrategyGetBuffer questions

2012-11-21 Thread Amit Kapila
in the buffer list, so that if some of the buffers are heavily used, then other backends might not need to pay penality for traversing Hot Buffers. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] [WIP PATCH] for Performance Improvement in Buffer Management

2012-11-22 Thread Amit Kapila
From: Pavan Deolasee [mailto:pavan.deola...@gmail.com] Sent: Thursday, November 22, 2012 12:26 PM To: Amit kapila Cc: Jeff Janes; pgsql-hackers@postgresql.org Subject: Re: [HACKERS] [WIP PATCH] for Performance Improvement in Buffer Management On Mon, Nov 19, 2012 at 8:52 PM, Amit

Re: [HACKERS] Switching timeline over streaming replication

2012-11-22 Thread Amit Kapila
On Wednesday, November 21, 2012 11:36 PM Heikki Linnakangas wrote: On 20.11.2012 15:33, Amit Kapila wrote: Defect-2: 1. start primary A 2. start standby B following A 3. start cascade standby C following B. 4. Start another standby D following C. 5. Execute

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-22 Thread Amit kapila
On Tuesday, November 20, 2012 7:21 PM Amit Kapila wrote: On Monday, November 19, 2012 9:07 PM Amit Kapila wrote: On Monday, November 19, 2012 8:36 PM Alvaro Herrera wrote: Amit Kapila escribió: The only point I can see against SET PERSISTENT is that other variants of SET command can

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-23 Thread Amit Kapila
On Thursday, November 22, 2012 10:09 PM Fujii Masao wrote: On Thu, Nov 22, 2012 at 9:38 PM, Amit kapila amit.kap...@huawei.com wrote: Patch to implement SET PERSISTENT command is attached with this mail. Now it can be reviewed. I got the following compile warnings: xact.c:1847: warning

Re: [HACKERS] [WIP PATCH] for Performance Improvement in Buffer Management

2012-11-23 Thread Amit Kapila
as well. http://blog.kimiensoftware.com/2011/05/postgresql-vs-oracle-differences-4-sh ared-memory-usage-257 So if above is true, I think the performance will regain if in the test Shared Buffers are set to 16G. I shall once try that setting for test run. With Regards, Amit Kapila.

Re: [HACKERS] [WIP PATCH] for Performance Improvement in Buffer Management

2012-11-23 Thread Amit kapila
On Friday, November 23, 2012 11:15 AM Pavan Deolasee wrote: On Thu, Nov 22, 2012 at 2:05 PM, Amit Kapila amit.kap...@huawei.com wrote: Sorry, I haven't followed this thread at all, but the numbers (43171 and 57920) in the last two runs of @mv-free-list for 32 clients look aberrations, no ? I

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-23 Thread Amit kapila
On Friday, November 23, 2012 10:10 PM Fujii Masao wrote: On Fri, Nov 23, 2012 at 6:56 PM, Amit Kapila amit.kap...@huawei.com wrote: When I remove postgresql.auto.conf, SET PERSISTENT failed. =# SET PERSISTENT synchronous_commit = 'local'; ERROR: failed to open postgresql.auto.conf file

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-24 Thread Amit kapila
On Saturday, November 24, 2012 10:56 PM Tom Lane wrote: Amit kapila amit.kap...@huawei.com writes: On Friday, November 23, 2012 10:10 PM Fujii Masao wrote: What happens if the server crashes while SET PERSISTENT is writing the setting to the file? A partial write occurs and restart

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Amit Kapila
suggestions? With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2012-11-26 Thread Amit Kapila
to the table space location instead of building the directory path. I am thinking if you follow the link instead, The utility can be used across the different versions of PG. This is good suggestion. I shall take care of this in updated patch. With Regards, Amit Kapila. Regards, Muhammad Usama

Re: Plugging fd leaks (was Re: [HACKERS] Switching timeline over streaming replication)

2012-11-26 Thread Amit Kapila
On Monday, November 26, 2012 7:01 PM Heikki Linnakangas wrote: On 26.11.2012 14:53, Amit Kapila wrote: I have one usecase in feature (SQL Command to edit postgresql.conf) very similar to OpenFile/CloseFile, but I want that when CloseFile is called from abort, it should remove(unlink

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2012-11-27 Thread Amit kapila
. Apart from above, updated the documentation. With Regards, Amit Kapila. pg_computemaxlsn.v2.patch Description: pg_computemaxlsn.v2.patch -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2012-11-27 Thread Amit Kapila
On Wednesday, November 28, 2012 3:36 AM Alvaro Herrera wrote: Amit Kapila escribió: Friday, November 23, 2012 5:38 AM Muhammad Usama wrote: - I think when finding the max LSN of single file the utility should consider all relation segments. Would you like to find for all relation

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2012-11-28 Thread Amit Kapila
On Wednesday, November 28, 2012 11:49 AM Muhammad Usama wrote: On Tue, Nov 27, 2012 at 5:52 PM, Amit kapila amit.kap...@huawei.com wrote: Friday, November 23, 2012 5:38 AM Muhammad Usama wrote:   - For -p {file | dir}  option the utility expects the file path relative to the specified data

Re: [HACKERS] Enabling frontend-only xlog desc routines

2012-11-28 Thread Amit Kapila
) need to define value for that particular Macro by using -D in makefile. If my above thinking is right, then I think Approach-2 might be better as in that Frontend will have more flexibility if it wants to use some other functionality of rmgr. With Regards, Amit Kapila. -- Sent via pgsql-hackers

Re: [HACKERS] Enabling frontend-only xlog desc routines

2012-11-28 Thread Amit Kapila
On Wednesday, November 28, 2012 7:07 PM Andres Freund wrote: On 2012-11-28 18:58:45 +0530, Amit Kapila wrote: On Wednesday, November 28, 2012 12:17 AM Alvaro Herrera wrote: I mentioned the remaining issues in a previous email (see message-id 20121025161751.ge6...@alvh.no-ip.org

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-29 Thread Amit Kapila
with it, but why to mark it as valid when actually there is no valid index. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Bugs in CREATE/DROP INDEX CONCURRENTLY

2012-11-29 Thread Amit Kapila
On Thursday, November 29, 2012 4:24 PM Andres Freund wrote: On 2012-11-29 16:18:07 +0530, Amit Kapila wrote: On Thursday, November 29, 2012 12:39 AM Tom Lane wrote. Andres Freund and...@2ndquadrant.com writes: On 2012-11-27 23:46:58 -0500, Tom Lane wrote: Attached is a very

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit Kapila
On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Wed, Nov 28, 2012 at 9:47 AM, Amit kapila amit.kap...@huawei.com wrote: 5. PERSISTENT Keyword is added to the reserved keyword list. As it was giving some errors given below while parsing

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit kapila
On Saturday, December 01, 2012 10:00 PM Tom Lane wrote: Amit Kapila amit.kap...@huawei.com writes : On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: But having said that, it's not apparent to me why inventing SET PERSISTENT should require reserving PERSISTENT. The problem is due

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-01 Thread Amit kapila
as it is and user will have mechanism to change default value only with SET PERSISTENT var_name TO DEFAULT. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-03 Thread Amit Kapila
On Saturday, December 01, 2012 10:00 PM Tom Lane wrote: Amit Kapila amit.kap...@huawei.com writes: On Saturday, December 01, 2012 1:30 AM Tom Lane wrote: which cannot work if persistent could be a var_name, because bison has to decide whether to reduce opt_persistent to PERSISTENT or empty

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2012-12-03 Thread Amit kapila
extendable in future. However I have no problem in implementing such functionality if you are of opinion that this is basic and it should go with first version of feature. With Regards, Amit Kapila. pg_computemaxlsn_v3.patch Description: pg_computemaxlsn_v3.patch -- Sent via pgsql-hackers mailing list

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-03 Thread Amit Kapila
in postgresql.auto.conf to default value However we can even change SET PERSISTENT... TO DEFAULT to delete the entry and then we can avoid RESET PERSISTENT ... Opinions? With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-03 Thread Amit Kapila
. How about if make the behavior of SET PERSISTENT... TO DEFAULT such that it will delete the entry in postgresql.auto.conf? With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] visibilitymap_count() at the end of vacuum

2012-12-04 Thread Amit Kapila
for interrupts there ? I think calling vacuum_delay_point(), after every visibility map bit test in lazy_scan_heap() might not be necessary. Shouldn't both places be consistent and call vacuum_delay_point() after one vm page processing? With Regards, Amit Kapila. -- Sent via pgsql-hackers

Re: [HACKERS] Switching timeline over streaming replication

2012-12-05 Thread Amit Kapila
a smart shutdown. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-05 Thread Amit Kapila
On Tuesday, December 04, 2012 8:37 AM Amit Kapila wrote: On Monday, December 03, 2012 8:59 PM Tom Lane wrote: Robert Haas robertmh...@gmail.com writes: On Sat, Dec 1, 2012 at 11:45 AM, Tom Lane t...@sss.pgh.pa.us wrote: But even if we can't make that work, it's not grounds for reserving

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-12-06 Thread Amit Kapila
. Vacuum the table 4. Upgrade the standby. 5. Select the all the tuples on new master 6. Vacuum the tbl on new master. 6B. Record the IO statistics time taken for Vacuum on new master. Suggestions/Feedback With Regards, Amit

Re: [HACKERS] [BUG?] lag of minRecoveryPont in archive recovery

2012-12-06 Thread Amit Kapila
, connections might not be allowed as CheckRecoveryConsistency() is not called. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Switching timeline over streaming replication

2012-12-06 Thread Amit Kapila
On Thursday, December 06, 2012 12:53 AM Heikki Linnakangas wrote: On 05.12.2012 14:32, Amit Kapila wrote: On Tuesday, December 04, 2012 10:01 PM Heikki Linnakangas wrote: After some diversions to fix bugs and refactor existing code, I've committed a couple of small parts of this patch

Re: [HACKERS] Setting visibility map in VACUUM's second phase

2012-12-06 Thread Amit Kapila
it can reduce some load of Vacuum as well, but the only thing is it should not make Page prune as heavy. By the way, isn't this idea similar to patch at below link: http://archives.postgresql.org/pgsql-hackers/2010-02/msg02344.php With Regards, Amit Kapila. -- Sent via pgsql-hackers

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2012-12-07 Thread Amit kapila
Hi Muhammad, On Friday, December 07, 2012 7:43 PM Muhammad Usama wrote: Hi Amit On Mon, Dec 3, 2012 at 6:56 PM, Amit kapila amit.kap...@huawei.commailto:amit.kap...@huawei.com wrote: I think we should expect provided path to be relative to current directory or may consider it to be relative

Re: [HACKERS] Review: Patch to compute Max LSN of Data Pages

2012-12-08 Thread Amit kapila
On Saturday, December 08, 2012 9:44 AM Tom Lane wrote: Amit kapila amit.kap...@huawei.com writes: On Friday, December 07, 2012 7:43 PM Muhammad Usama wrote: Although I am thinking why are you disallowing the absolute path of file. Any particular reason? The reason to disallow absolute path

Re: [HACKERS] CommitFest #3 and upcoming schedule

2012-12-09 Thread Amit Kapila
if I should attach myself as reviewer to more patches as per initial policy of CF? In anycase as soon as I get time I shall review more patches. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http

Re: [HACKERS] [BUG?] lag of minRecoveryPont in archive recovery

2012-12-10 Thread Amit Kapila
(); LWLockRelease(ControlFileLock); } CheckRecoveryConsistency(); This had completely resolved the problem reported on above link for me. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref

Re: [HACKERS] Switching timeline over streaming replication

2012-12-10 Thread Amit Kapila
From: Heikki Linnakangas [mailto:hlinnakan...@vmware.com] Sent: Friday, December 07, 2012 9:22 PM To: Amit Kapila Cc: 'PostgreSQL-development'; 'Thom Brown' Subject: Re: [HACKERS] Switching timeline over streaming replication On 06.12.2012 15:39, Amit Kapila wrote: On Thursday, December

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-12-11 Thread Amit Kapila
On Tuesday, December 11, 2012 5:25 AM Jaime Casanova wrote: On Fri, Nov 23, 2012 at 4:56 AM, Amit Kapila amit.kap...@huawei.com wrote: On Thursday, November 22, 2012 10:09 PM Fujii Masao wrote: Is it helpful to output the notice message like 'Run pg_reload_conf() or restart the server

Re: [HACKERS] [WIP PATCH] for Performance Improvement in Buffer Management

2012-12-12 Thread Amit Kapila
On Wednesday, December 12, 2012 5:23 AM Greg Smith wrote: On 11/23/12 5:57 AM, Amit kapila wrote: Let us try to see by example: Total RAM - 22G Database size - 16G ... Case -2 (Shared Buffers - 10G) a. Load all the files in OS buffers. In best case OS buffers can contain10-12G data

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2012-12-14 Thread Amit Kapila
. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP patch for hint bit i/o mitigation

2012-12-14 Thread Amit Kapila
is that in the test pages are getting dirty only due to setting of hint bit by Vacuum. -- I'd like to see the bulk insert performance hit reduced if possible. I think if we can improve performance for bulk-insert case, then this patch has much more value. With Regards, Amit Kapila. -- Sent via pgsql

Re: [HACKERS] Set visibility map bit after HOT prune

2012-12-18 Thread Amit Kapila
such that new value is always increasing, then index bloat happens. As per initial analysis, it seems one of the reasons is what I described above. If required, I can create a self-containing test which can show that bulk-index update can lead to index bloat. With Regards, Amit Kapila. -- Sent

[HACKERS] Review : Add hooks for pre- and post-processor executables for COPY and \copy

2012-12-18 Thread Amit Kapila
. decompress.sh echo 'bzip2 -d -c -k $*' decompress.sh chmod +x decompress.sh Testcases executed are attached with this mail. With Regards, Amit Kapila. --1. check success case admin can use this COPY statement feature. select count(*) from pgbench_accounts; copy

Re: [HACKERS] Feature Request: pg_replication_master()

2012-12-19 Thread Amit Kapila
that planned for SET PERSISTENT? Yes in SET PERSISTENT patch, it uses include_dir. I wonder why can't we get this information from WALRcvData structure? It has the required information. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Set visibility map bit after HOT prune

2012-12-19 Thread Amit Kapila
taking X lock and then during split (when it already has X lock) it free's the actual space. So not sure if it's good idea to take X lock for cleanup during heap scan, where write operation's happens more frequently and have better chance of cleanup. With Regards, Amit Kapila. -- Sent via pgsql

Re: [HACKERS] ThisTimeLineID in checkpointer and bgwriter processes

2012-12-20 Thread Amit Kapila
problem. I think it could get called from CreateRestartPoint() during recovery. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] ThisTimeLineID in checkpointer and bgwriter processes

2012-12-20 Thread Amit Kapila
On Thursday, December 20, 2012 5:12 PM Heikki Linnakangas wrote: On 20.12.2012 12:08, Amit Kapila wrote: On Wednesday, December 19, 2012 9:30 PM Heikki Linnakangas wrote: In both checkpointer.c and bgwriter.c, we do this before entering the main loop: /* * Use

Re: [HACKERS] ThisTimeLineID in checkpointer and bgwriter processes

2012-12-20 Thread Amit Kapila
On Thursday, December 20, 2012 7:03 PM Simon Riggs wrote: On 20 December 2012 13:19, Amit Kapila amit.kap...@huawei.com wrote: So I think we're good on that front. But I'll add a comment there, and use 0 explicitly instead of ThisTimeLineID, for clarity. True, it might not have any

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2012-12-20 Thread Amit Kapila
On Thursday, December 20, 2012 5:46 PM Simon Riggs wrote: On 13 October 2012 08:54, Amit kapila amit.kap...@huawei.com wrote: As per the last discussion for this patch, performance data needs to be provided before this patch's Review can proceed further. So as per your suggestion

Re: [HACKERS] ThisTimeLineID in checkpointer and bgwriter processes

2012-12-20 Thread Amit Kapila
if(RecoveryInProgress), before assigning RecoveryTargetTLI to ThisTimeLineID in CreateRestartPoint()? With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_basebackup from cascading standby after timeline switch

2012-12-21 Thread Amit kapila
? With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2012-12-24 Thread Amit Kapila
On Sunday, December 23, 2012 8:11 PM Simon Riggs wrote: On 20 December 2012 14:56, Amit Kapila amit.kap...@huawei.com wrote: 1. There is no performance change for cloumns that have all valid values(non- NULLs). I don't see any tests (at all) that measure this. I'm particularly

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2012-12-24 Thread Amit Kapila
to be run. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Re: patch submission: truncate trailing nulls from heap rows to reduce the size of the null bitmap [Review]

2012-12-24 Thread Amit Kapila
of 3 runs of pgbench in tps 9.3devel | with trailing null patch --+-- 578.9872 | 573.4980 With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2012-12-28 Thread Amit Kapila
better modularity, so the value itself needn't be exported I shall update the patch to address it. * Need to mention the WAL format change, or include the change within the patch so we can see Sure, I will update this in code comments and internals docs. With Regards, Amit Kapila

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2012-12-28 Thread Amit Kapila
of WAL devices Thank you very much for the review. With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Performance Improvement by reducing WAL for Update Operation

2013-01-04 Thread Amit Kapila
, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-01-04 Thread Amit kapila
about reducing the time of sleep or removing sleep, in that user might get error and he need to retry to get his command successful? With Regards, Amit Kapila. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL [review]

2013-01-05 Thread Amit kapila
On Saturday, January 05, 2013 12:35 PM Boszormenyi Zoltan wrote: 2013-01-05 05:58 keltezéssel, Amit kapila írta: On Friday, January 04, 2013 10:57 PM Boszormenyi Zoltan wrote: Hi, I am reviewing your patch. Thank you very much. Yes, you are right adding a new LWLock will avoid the use

[HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-07 Thread Amit Kapila
something? With Regards, Amit Kapila.

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-07 Thread Amit Kapila
On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January 2013 12:39, Amit Kapila amit.kap...@huawei.com wrote: So We can modify to change this in function LogStandbySnapshot as below: running = GetRunningTransactionData(); if (running-xcnt 0

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Amit Kapila
On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January 2013 12:39, Amit Kapila amit.kap...@huawei.com wrote: So We can modify to change this in function

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-08 Thread Amit Kapila
On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01-07 19:03:35 +0530, Amit Kapila wrote: On Monday, January 07, 2013 6:30 PM Simon Riggs wrote: On 7 January

Re: [HACKERS] Extra XLOG in Checkpoint for StandbySnapshot

2013-01-09 Thread Amit Kapila
On Tuesday, January 08, 2013 8:57 PM Andres Freund wrote: On 2013-01-08 20:33:28 +0530, Amit Kapila wrote: On Tuesday, January 08, 2013 8:01 PM Andres Freund wrote: On 2013-01-08 19:51:39 +0530, Amit Kapila wrote: On Monday, January 07, 2013 7:15 PM Andres Freund wrote: On 2013-01

  1   2   3   4   5   6   7   8   9   10   >