Re: [HACKERS] logical changeset generation v3

2012-11-18 Thread Michael Paquier
Hi Andres, I have been able to fetch your code (thanks Andrea!) and some it. For the time being I am spending some time reading the code and understanding the whole set of features you are trying to implement inside core, even if I got some background from what you presented at PGCon and from the

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-18 Thread Amit Kapila
On Sunday, November 18, 2012 3:22 PM Cédric Villemain wrote: > Le samedi 17 novembre 2012 22:57:49, Tom Lane a écrit : > > Fujii Masao writes: > > > Do we really need to store the settings in a system table? > > > Since WAL would be generated when storing the settings in a system > > > table, this

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-18 Thread Amit Kapila
On Sunday, November 18, 2012 3:28 AM Tom Lane wrote: > Fujii Masao writes: > > Do we really need to store the settings in a system table? > > Since WAL would be generated when storing the settings > > in a system table, this approach seems to prevent us from > > changing the settings in the standb

Re: [HACKERS] [PATCH 13/14] Introduce pg_receivellog, the pg_receivexlog equivalent for logical changes

2012-11-18 Thread Michael Paquier
Hi, I am just looking at this patch and will provide some comments. By the way, you forgot the installation part of pg_receivellog, please see patch attached. Thanks, On Thu, Nov 15, 2012 at 10:17 AM, Andres Freund wrote: > --- > src/bin/pg_basebackup/Makefile | 7 +- > src/bin/pg_bas

Re: [HACKERS] pg_trgm partial-match

2012-11-18 Thread Alexander Korotkov
Hi! On Thu, Nov 15, 2012 at 11:39 PM, Fujii Masao wrote: > Note that we cannot do a partial-match if KEEPONLYALNUM is disabled, > i.e., if query key contains multibyte characters. In this case, byte > length of > the trigram string might be larger than three, and its CRC is used as a > trigram k

[HACKERS] [PATCH] Patch to fix missing libecpg_compat.lib and libpgtypes.lib.

2012-11-18 Thread JiangGuiqing
hi I install postgresql-9.1.5 from source code on windows successfully. But there is not "libecpg_compat.lib" and "libpgtypes.lib" in the installed lib directory . If someone used functions of pgtypes or ecpg_compat library in ecpg programs,the ecpg program build will fail. So i modify src/tools/

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-18 Thread Amit Kapila
> On Sunday, November 18, 2012 3:08 AM Fujii Masao wrote: > On Sat, Nov 17, 2012 at 10:25 PM, Amit Kapila > wrote: > > 1. have a system table pg_global_system_settings(key,value) > > Do we really need to store the settings in a system table? > Since WAL would be generated when storing the setting

Re: [HACKERS] pg_trgm partial-match

2012-11-18 Thread Tomas Vondra
On 15.11.2012 20:39, Fujii Masao wrote: > Hi, > > I'd like to propose to extend pg_trgm so that it can compare a partial-match > query key to a GIN index. IOW, I'm thinking to implement the 'comparePartial' > GIN method for pg_trgm. > > Currently, when the query key is less than three characters,

Re: [HACKERS] [RFC] Fix div/mul crash and more undefined behavior

2012-11-18 Thread Xi Wang
On 11/18/12 6:47 PM, Tom Lane wrote: > Xi Wang writes: >> [ patch adding a bunch of explicit INT_MIN/MAX constants ] > > I was against this style of coding before, and I still am. > For one thing, it's just about certain to introduce conflicts > against system headers. I totally agree. I would

Re: [HACKERS] [WIP] pg_ping utility

2012-11-18 Thread Michael Paquier
On Fri, Nov 16, 2012 at 2:28 PM, Tom Lane wrote: > Maybe I missed something here, but I believe it's standard that > "program --help" should result in exit(0), no matter what the program's > exitcode conventions are for live-fire exercises. > Yes indeed you are right. Thanks. -- Michael Paquier

Re: [HACKERS] [WIP] pg_ping utility

2012-11-18 Thread Michael Paquier
On Sat, Nov 17, 2012 at 2:48 AM, Phil Sorber wrote: > On Thu, Nov 15, 2012 at 10:55 PM, Michael Paquier > wrote: > > On Fri, Nov 16, 2012 at 12:34 PM, Phil Sorber wrote: > >> On Thu, Nov 15, 2012 at 9:23 PM, Michael Paquier > >> wrote: > >> > 3) Having an output close to what ping actually doe

Re: [HACKERS] [WIP PATCH] for Performance Improvement in Buffer Management

2012-11-18 Thread Jeff Janes
On Mon, Oct 22, 2012 at 10:51 AM, Amit kapila wrote: > > Today again I have again collected the data for configuration Shared_buffers > = 7G along with vmstat. > The data and vmstat information (bi) are attached with this mail. It is > observed from vmstat info that I/O is happening for both ca

Re: [HACKERS] logical changeset generation v3

2012-11-18 Thread Michael Paquier
On Fri, Nov 16, 2012 at 5:16 PM, Andrea Suisani wrote: > Il 16/11/2012 05:34, Michael Paquier ha scritto: > > Do you have a git repository or something where all the 14 patches are >> applied? I would like to test the feature globally. >> Sorry I recall that you put a link somewhere but I cannot

Re: [HACKERS] [WIP PATCH] for Performance Improvement in Buffer Management

2012-11-18 Thread Jeff Janes
On Sun, Oct 21, 2012 at 12:59 AM, Amit kapila wrote: > On Saturday, October 20, 2012 11:03 PM Jeff Janes wrote: > >>Run the modes in reciprocating order? > Sorry, I didn't understood this, What do you mean by modes in reciprocating > order? Sorry for the long delay. In your scripts, it looks li

Re: [HACKERS] Parser - Query Analyser

2012-11-18 Thread Craig Ringer
On 11/18/2012 09:57 PM, Michael Giannakopoulos wrote: > Hi guys, > > Thanks for your answers. Yes, what I meant is to create a function > that takes as an input rows of a specific relation, does something and > returns as an output rows with different attributes. I am > experimenting right now with

Re: [HACKERS] 9.3 pg_archivecleanup broken?

2012-11-18 Thread Fujii Masao
On Mon, Nov 19, 2012 at 12:43 AM, Erik Rijkers wrote: > (In a test setup) I can't get pg_archivecleanup to remove WALfiles in > 9.3devel. (A very similar > setup in 9.2 works fine). > > In 9.3 pg_archivecleanup just keeps repeating lines like: > > pg_archivecleanup: keep WAL file > "/home/aardva

Re: [HACKERS] [RFC] Fix div/mul crash and more undefined behavior

2012-11-18 Thread Tom Lane
Xi Wang writes: > [ patch adding a bunch of explicit INT_MIN/MAX constants ] I was against this style of coding before, and I still am. For one thing, it's just about certain to introduce conflicts against system headers. regards, tom lane -- Sent via pgsql-hackers mai

[HACKERS] [RFC] Fix div/mul crash and more undefined behavior

2012-11-18 Thread Xi Wang
The INT_MIN / -1 crash problem was partially addressed in 2006 and commit 9fc6f4e1ae107f44807c4906105e1f7eb052ecb1. http://archives.postgresql.org/pgsql-patches/2006-06/msg00102.php However, the fix is incomplete and incorrect for some cases. 64-bit crash Below is an example that c

Re: [HACKERS] autovacuum stress-testing our system

2012-11-18 Thread Tomas Vondra
On 26.9.2012 18:29, Tom Lane wrote: > What seems to me like it could help more is fixing things so that the > autovac launcher needn't even launch a child process for databases that > haven't had any updates lately. I'm not sure how to do that, but it > probably involves getting the stats collecto

Re: [HACKERS] pg_dump --split patch

2012-11-18 Thread Marko Tiikkaja
Hi, On 16/11/2012 15:52, Dimitri Fontaine wrote: Marko Tiikkaja writes: The general output scheme looks like this: schemaname/OBJECT_TYPES/object_name.sql, I like this feature, I actually did have to code it myself in the past and several other people did so, so we already have at least

Re: [HACKERS] autovacuum stress-testing our system

2012-11-18 Thread Tomas Vondra
Hi! On 26.9.2012 19:18, Jeff Janes wrote: > On Wed, Sep 26, 2012 at 9:29 AM, Tom Lane wrote: >> Alvaro Herrera writes: >>> Excerpts from Euler Taveira's message of mié sep 26 11:53:27 -0300 2012: On 26-09-2012 09:43, Tomas Vondra wrote: > 5) splitting the single stat file into multiple

[HACKERS] WIP: store additional info in GIN index

2012-11-18 Thread Alexander Korotkov
Hackers, Attached patch enables GIN to store additional information with item pointers in posting lists and trees. Such additional information could be positions of words, positions of trigrams, lengths of arrays and so on. This is the first and most huge patch of serie of GIN improvements which w

Re: [HACKERS] Avoiding overflow in timeout-related calculations

2012-11-18 Thread Andres Freund
On 2012-11-18 15:21:34 -0500, Tom Lane wrote: > Andres Freund writes: > > I think at least wal_sender_timeout and wal_receiver_timeout are also > > problematic. > > I looked at those and didn't see a problem --- what are you worried > about exactly? Forget it, too hungry to read the code properl

Re: [HACKERS] Avoiding overflow in timeout-related calculations

2012-11-18 Thread Tom Lane
Andres Freund writes: > I think at least wal_sender_timeout and wal_receiver_timeout are also > problematic. I looked at those and didn't see a problem --- what are you worried about exactly? regards, tom lane -- Sent via pgsql-hackers mailing list (pgsql-hackers@postg

Re: [HACKERS] Avoiding overflow in timeout-related calculations

2012-11-18 Thread Andres Freund
On 2012-11-18 14:57:51 -0500, Tom Lane wrote: > The discussion of bug #7670 showed that what's happening there is that > if you specify a log_rotation_age of more than 25 days (2^31 msec), > WaitLatch will sometimes be passed a timeout of more than 2^31 msec, > leading to unportable behavior. At l

Re: [HACKERS] Enabling Checksums

2012-11-18 Thread Jeff Davis
ormat for checksums, so we shouldn't defer decisions that would affect that (e.g. doing checksum calculation in larger chunks, ignoring the page hole, or using a different scheme for the bits in the header). Regards, Jeff Davis replace-tli-with-checksums-20121118.patch.gz Descr

[HACKERS] Avoiding overflow in timeout-related calculations

2012-11-18 Thread Tom Lane
The discussion of bug #7670 showed that what's happening there is that if you specify a log_rotation_age of more than 25 days (2^31 msec), WaitLatch will sometimes be passed a timeout of more than 2^31 msec, leading to unportable behavior. At least some kernels will return EINVAL for that, and it'

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby [Review]

2012-11-18 Thread Boszormenyi Zoltan
Hi, 2012-11-18 17:20 keltezéssel, Magnus Hagander írta: On Tue, Oct 23, 2012 at 5:08 PM, Magnus Hagander wrote: On Oct 23, 2012 4:52 PM, "Alvaro Herrera" wrote: Boszormenyi Zoltan escribió: Also, the check for conflict between -R and -x/-X is now removed. The documentation for option -R h

Re: [HACKERS] Do we need so many hint bits?

2012-11-18 Thread Jeff Davis
On Sun, 2012-11-18 at 15:19 +0100, Andres Freund wrote: > On Sunday, November 18, 2012 03:07:01 AM Jeff Davis wrote: > > Process A (process that clears a VM bit for a data page): > > 1. Acquires exclusive lock on data buffer > > 2. Acquires exclusive lock on VM buffer > > 3. clears VM bit > >

Re: [HACKERS] [PATCH] Make pg_basebackup configure and start standby [Review]

2012-11-18 Thread Magnus Hagander
On Tue, Oct 23, 2012 at 5:08 PM, Magnus Hagander wrote: > > On Oct 23, 2012 4:52 PM, "Alvaro Herrera" wrote: >> >> Boszormenyi Zoltan escribió: >> >> > >>Also, the check for conflict between -R and -x/-X is now removed. >> > >> > The documentation for option -R has changed to reflect this and >>

[HACKERS] Re: [PATCH 05/14] Add a new relmapper.c function RelationMapFilenodeToOid that acts as a reverse of RelationMapOidToFilenode

2012-11-18 Thread Andres Freund
On 2012-11-17 19:14:06 +0900, Michael Paquier wrote: > On Fri, Nov 16, 2012 at 7:58 PM, Andres Freund wrote: > > > Hi, > > > > On 2012-11-16 13:44:45 +0900, Michael Paquier wrote: > > > This patch looks OK. > > > > > > I got 3 comments: > > > 1) Why changing the OID of pg_class_tblspc_relfilenode_

[HACKERS] 9.3 pg_archivecleanup broken?

2012-11-18 Thread Erik Rijkers
(In a test setup) I can't get pg_archivecleanup to remove WALfiles in 9.3devel. (A very similar setup in 9.2 works fine). In 9.3 pg_archivecleanup just keeps repeating lines like: pg_archivecleanup: keep WAL file "/home/aardvark/pg_stuff/archive_dir93/" and later (and

Re: [HACKERS] Do we need so many hint bits?

2012-11-18 Thread Simon Riggs
On 18 November 2012 08:52, Simon Riggs wrote: > The difference here is that we still need to check visibility of each > tuple, but that can be a very cheap check and never involves clog, nor > does it dirty the page. Tuple access is reasonably expensive in > comparison with a clog-less check on t

Re: [HACKERS] Do we need so many hint bits?

2012-11-18 Thread Andres Freund
On Sunday, November 18, 2012 03:07:01 AM Jeff Davis wrote: > Process A (process that clears a VM bit for a data page): > 1. Acquires exclusive lock on data buffer > 2. Acquires exclusive lock on VM buffer > 3. clears VM bit > 4. Releases VM buffer lock > 5. Releases data buffer lock Well

Re: [HACKERS] Do we need so many hint bits?

2012-11-18 Thread Simon Riggs
On 17 November 2012 21:20, Jeff Davis wrote: >> ISTM that we should tune that specifically by performing a VM lookup >> for next 32 pages (or more), so we reduce the lookups well below 1 per >> page. That way the overhead of using the VM will be similar to using >> the PD_ALL_VISIBLE. > > That's

Re: [HACKERS] Parser - Query Analyser

2012-11-18 Thread Craig Ringer
On 11/17/2012 10:18 PM, Michael Giannakopoulos wrote: > Hello guys, > > My name is Michail Giannakopoulos and I am a graduate student at > University of Toronto. I have no previous experience in developing a > system like postgreSQL before. > > What I am trying to explore is if it is possible to ex

Re: [HACKERS] Materialized views WIP patch

2012-11-18 Thread Simon Riggs
On 16 November 2012 11:25, Kevin Grittner wrote: >>> 16. To get new data into the MV, the command is LOAD MATERIALIZED >>> VIEW mat view_name. This seemed more descriptive to me that the >>> alternatives and avoids declaring any new keywords beyond >>> MATERIALIZED. If the MV is flagged as relisv

Re: [HACKERS] Proposal for Allow postgresql.conf values to be changed via SQL

2012-11-18 Thread Cédric Villemain
Le samedi 17 novembre 2012 22:57:49, Tom Lane a écrit : > Fujii Masao writes: > > Do we really need to store the settings in a system table? > > Since WAL would be generated when storing the settings > > in a system table, this approach seems to prevent us from > > changing the settings in the sta