Re: [HACKERS] Hooks to track changed pages for backup purposes

> On 13 Sep 2017, at 15:01, Tomas Vondra wrote: > > On 09/13/2017 07:53 AM, Andrey Borodin wrote: >>> * I see there are conditions like this: >>> >>>if(xlogreader->blocks[nblock].forknum == MAIN_FORKNUM) >>> >>> Why is it enough to restrict the block-tracking

Re: [HACKERS] Hooks to track changed pages for backup purposes

On 09/13/2017 07:53 AM, Andrey Borodin wrote: >> * I see there are conditions like this: >> >>    if(xlogreader->blocks[nblock].forknum == MAIN_FORKNUM) >> >> Why is it enough to restrict the block-tracking code to main fork? >> Aren't we interested in all relation forks? > fsm, vm and others

Re: [HACKERS] Hooks to track changed pages for backup purposes

Hi! Thank you for your interest and experiment results. > 13 сент. 2017 г., в 15:43, Ants Aasma написал(а): > > On Thu, Aug 31, 2017 at 9:02 AM, Andrey Borodin wrote: >> When we have accumulated diff blocknumbers for most of segments we can >>

Re: [HACKERS] Hooks to track changed pages for backup purposes

On Thu, Aug 31, 2017 at 9:02 AM, Andrey Borodin wrote: > When we have accumulated diff blocknumbers for most of segments we can > significantly speed up method of WAL scanning. If we have blocknumbers for > all segments we can skip WAL scanning at all. Have you measured

Re: [HACKERS] Hooks to track changed pages for backup purposes

Hi Tomas! Thank you for looking into that patch. > 8 сент. 2017 г., в 1:53, Tomas Vondra > написал(а): > > A few more comments: > > * The patch defines wal_switch_hook, but it's never called. That call was missing, that's a bug, thanks for spotting that out. > *

Re: [HACKERS] Hooks to track changed pages for backup purposes

Hi, On 09/01/2017 08:13 AM, Andrey Borodin wrote: > Thank you for your reply, Michael! Your comments are valuable, especially in the world of backups. > >> 31 авг. 2017 г., в 19:44, Michael Paquier написал(а): >> Such things are not Postgres-C like. > Will be fixed. >

Re: [HACKERS] Hooks to track changed pages for backup purposes

Thank you for your reply, Michael! Your comments are valuable, especially in the world of backups. > 31 авг. 2017 г., в 19:44, Michael Paquier > написал(а): > Such things are not Postgres-C like. Will be fixed. > I don't understand what xlog_begin_insert_hook() is

Re: [HACKERS] Hooks to track changed pages for backup purposes

On Thu, Aug 31, 2017 at 3:02 PM, Andrey Borodin wrote: > Here is the patch with hooks that I consider sufficient for implementation of > incremental backup with pages tracking as extension. > > Recently I was posting these things to the thread "Adding hook in BufferSync >

[HACKERS] Hooks to track changed pages for backup purposes

Hi hackers! Here is the patch with hooks that I consider sufficient for implementation of incremental backup with pages tracking as extension. Recently I was posting these things to the thread "Adding hook in BufferSync for backup purposes" [0], but here I start separate thread since Subj

Re: [HACKERS] Hooks

On 12/28/16 10:43 AM, David Fetter wrote: Callbacks aren't easy to find either. Should callbacks be another chapter in the docs? That would also be nice, but I suspect that will be harder than finding all the hooks. -- Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX Experts in

Re: [HACKERS] Hooks

On 12/27/16 11:14 PM, David Fetter wrote: Sure, but that seems like an effort somewhat orthogonal to the one I proposed, which is to get some user-facing i.e. SGML docs up for the current hooks. My point was that a (presumably small) amount of effort towards earmarking hooks in code so that a

Re: [HACKERS] Hooks

On Wed, Dec 28, 2016 at 12:19:11PM +0800, Craig Ringer wrote: > On 28 December 2016 at 12:15, Jim Nasby wrote: > > > Can we reduce the scope of this to a manageable starting point? > > I'm guessing that all existing hooks share certain characteristics > > that it'd be

Re: [HACKERS] Hooks

On Wed, Dec 28, 2016 at 03:07:52PM +0900, Michael Paquier wrote: > On Wed, Dec 28, 2016 at 2:14 PM, David Fetter wrote: > > Here's everything that matches ^\w+_hook$ that I've found so far in > > git master. There are very likely false positives in this list. > > > > [... long

Re: [HACKERS] Hooks

On Wed, Dec 28, 2016 at 2:14 PM, David Fetter wrote: > Here's everything that matches ^\w+_hook$ that I've found so far in > git master. There are very likely false positives in this list. > > [... long list of hooks ...] > > Some appear to be client-side, some server-side. I

Re: [HACKERS] Hooks

On Tue, Dec 27, 2016 at 10:15:55PM -0600, Jim Nasby wrote: > On 12/27/16 7:41 PM, David Fetter wrote: > > I see it as larger in scope than mine because it changes how we do > > things as a project. An example of the kind of thing that this raises > > is enforcement. Will something (or someone)

Re: [HACKERS] Hooks

On 28 December 2016 at 12:15, Jim Nasby wrote: > Can we reduce the scope of this to a manageable starting point? I'm guessing > that all existing hooks share certain characteristics that it'd be pretty > easy to detect. If you can detect the hook (which I guess means

Re: [HACKERS] Hooks

On 12/27/16 7:41 PM, David Fetter wrote: I see it as larger in scope than mine because it changes how we do things as a project. An example of the kind of thing that this raises is enforcement. Will something (or someone) check that new hooks have this? Will somebody check for comment skew

Re: [HACKERS] Hooks

On Wed, Dec 28, 2016 at 01:33:13AM +, Tsunakawa, Takayuki wrote: > From: David Fetter [mailto:da...@fetter.org] > > > How about putting a descriptive comment at the location where each > > > hook variable is defined, using some convention (e.g. like > > > Javadoc-style)? A separate document

Re: [HACKERS] Hooks

From: David Fetter [mailto:da...@fetter.org] > > How about putting a descriptive comment at the location where each > > hook variable is defined, using some convention (e.g. like > > Javadoc-style)? A separate document such as README and wiki can fail > > to be updated. OTOH, if someone wants to

Re: [HACKERS] Hooks

On Wed, Dec 28, 2016 at 12:47:10AM +, Tsunakawa, Takayuki wrote: > From: pgsql-hackers-ow...@postgresql.org > > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Jim Nasby > > AFAIK there's no way to get a list of hooks today, short of > > something like `git grep hook`. I think a

Re: [HACKERS] Hooks

From: pgsql-hackers-ow...@postgresql.org > [mailto:pgsql-hackers-ow...@postgresql.org] On Behalf Of Jim Nasby > AFAIK there's no way to get a list of hooks today, short of something like > `git grep hook`. I think a simple list of what hooks we have, when they > fire and where to find them in code

Re: [HACKERS] Hooks

On 12/27/16 1:43 PM, David Fetter wrote: So I'm a bit suspicious of this project in the first place, but it's hard to discuss which hooks should be documented when you haven't defined what you mean by documentation. I haven't quite come up with that, but I'd pictured a part of the SGML docs

Re: [HACKERS] Hooks

On Tue, Dec 27, 2016 at 01:32:46PM -0500, Tom Lane wrote: > David Fetter writes: > > One of our hidden treasures is the hook system, documented only in > > random presentations, if you can find them, and in the source code, if > > you know to look. > > > I'd like to document

Re: [HACKERS] Hooks

David Fetter writes: > One of our hidden treasures is the hook system, documented only in > random presentations, if you can find them, and in the source code, if > you know to look. > I'd like to document the hooks that we consider public APIs. The main reason we send people

[HACKERS] Hooks

Folks, One of our hidden treasures is the hook system, documented only in random presentations, if you can find them, and in the source code, if you know to look. I'd like to document the hooks that we consider public APIs. To do this, I need to figure out whether there are hooks that we don't

[HACKERS] Hooks Docu - list of hooks

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi to all! I am searching for a documentation of hooks in PG, all i found was a presentation in the wiki and some modules from 2ndQuadrant and petere on github. The last three weeks i was reading the source code to get some information. Is there a

Re: [HACKERS] Hooks Docu - list of hooks

At 2014-06-24 12:49:17 +0200, li...@hasibether.at wrote: Is there a list of possible hooks, or maybe a little docu or overview? The best I found was git grep _hook_type and then read the code to understand when and why the hook was called. Especially hooks to catch Insert, Update and Delete

Re: [HACKERS] Hooks Docu - list of hooks

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 24/06/14 12:59, Abhijit Menon-Sen wrote: At 2014-06-24 12:49:17 +0200, li...@hasibether.at wrote: Is there a list of possible hooks, or maybe a little docu or overview? The best I found was git grep _hook_type and then read the code to

[HACKERS] hooks not working in postgresql 9.3 (building from source)

hi, I need to use the hook feature of planner in postgresql 9.3 to perform a task. I am building postgresql from source. To start with, I wanted to know how to create a hook and attach that shared libray to postgresql. Hence i tried the examples given in

Re: [HACKERS] hooks not working in postgresql 9.3 (building from source)

On Thu, May 1, 2014 at 12:33 AM, Rajmohan C csrajmo...@gmail.com wrote: I need to use the hook feature of planner in postgresql 9.3 to perform a task. I am building postgresql from source. This hook is called planner_hook_type and is present in planner.c. To start with, I wanted to know how

Re: [HACKERS] HOOKS for Synchronous Replication?

I fixed the patch and now it compiles in windows. The first one did not compile because there were some problems in the Makefile. It is currently available for download at http://gorda.di.uminho.pt/community/pgsqlhooks/. Regards, Alfranio ---(end of

Re: [HACKERS] HOOKS for Synchronous Replication?

I can only add that patched code did not build on windows, contacted author about that and never got an answer back. Regards paolo Tom Lane [EMAIL PROTECTED] ha scritto Christopher Kings-Lynne [EMAIL PROTECTED] writes: Anyone remember this patch?

Re: [HACKERS] HOOKS for Synchronous Replication?

On Wednesday 07 December 2005 20:24, Tom Lane wrote: Christopher Kings-Lynne [EMAIL PROTECTED] writes: Anyone remember this patch? http://gorda.di.uminho.pt/community/pgsqlhooks/ The discussion seems to be pretty minimal: http://archives.postgresql.org/pgsql-hackers/2005-06/msg00859.php

Re: [HACKERS] HOOKS for Synchronous Replication?

On Thu, Dec 08, 2005 at 08:33:59AM -0800, Darcy Buskermolen wrote: On Wednesday 07 December 2005 20:24, Tom Lane wrote: Christopher Kings-Lynne [EMAIL PROTECTED] writes: Anyone remember this patch? http://gorda.di.uminho.pt/community/pgsqlhooks/ The discussion seems to be pretty

Re: [HACKERS] HOOKS for Synchronous Replication?

On 12/7/2005 11:24 PM, Tom Lane wrote: Christopher Kings-Lynne [EMAIL PROTECTED] writes: Anyone remember this patch? http://gorda.di.uminho.pt/community/pgsqlhooks/ The discussion seems to be pretty minimal: http://archives.postgresql.org/pgsql-hackers/2005-06/msg00859.php Does anyone see a

Re: [HACKERS] HOOKS for Synchronous Replication?

On 12/8/05, Jim C. Nasby [EMAIL PROTECTED] wrote: While this code might be useful, whouldn't it be much more valuable toprovide hooks into xlog so that we could do non-trigger-basedreplication? (As well as non-trigger-based materialized views...) If we're going to do hooks for replication, I

Re: [HACKERS] HOOKS for Synchronous Replication?

; Tom Lane; Christopher Kings-Lynne Subject: Re: [HACKERS] HOOKS for Synchronous Replication? On 12/8/05, Jim C. Nasby [EMAIL PROTECTED] wrote: While this code might be useful, whouldn't it be much more valuable to provide hooks into xlog so that we could do non-trigger-based replication

Re: [HACKERS] HOOKS for Synchronous Replication?

On 12/8/2005 2:05 PM, Jim C. Nasby wrote: On Thu, Dec 08, 2005 at 08:33:59AM -0800, Darcy Buskermolen wrote: On Wednesday 07 December 2005 20:24, Tom Lane wrote: Christopher Kings-Lynne [EMAIL PROTECTED] writes: Anyone remember this patch? http://gorda.di.uminho.pt/community/pgsqlhooks/

Re: [HACKERS] HOOKS for Synchronous Replication?

True, I think we need hooks for both methods.On 12/8/05, Jan Wieck [EMAIL PROTECTED] wrote: On 12/8/2005 2:05 PM, Jim C. Nasby wrote: On Thu, Dec 08, 2005 at 08:33:59AM -0800, Darcy Buskermolen wrote: On Wednesday 07 December 2005 20:24, Tom Lane wrote: Christopher Kings-Lynne [EMAIL PROTECTED]

[HACKERS] HOOKS for Synchronous Replication?

Anyone remember this patch? http://gorda.di.uminho.pt/community/pgsqlhooks/ The discussion seems to be pretty minimal: http://archives.postgresql.org/pgsql-hackers/2005-06/msg00859.php Does anyone see a need to investigate it further? Chris ---(end of

Re: [HACKERS] HOOKS for Synchronous Replication?

Christopher Kings-Lynne [EMAIL PROTECTED] writes: Anyone remember this patch? http://gorda.di.uminho.pt/community/pgsqlhooks/ The discussion seems to be pretty minimal: http://archives.postgresql.org/pgsql-hackers/2005-06/msg00859.php Does anyone see a need to investigate it further? I had

Re: [HACKERS] HOOKS for Synchronous Replication

Alfranio Correia Junior wrote: I think it is ok now. However, I corrected the indentation manually. I could not run some of the tools, namely the entab. /usr/lib/gcc-lib/i386-redhat-linux/3.3.3/include/varargs.h:4:2: #error GCC no longer implements varargs.h.

Re: [HACKERS] HOOKS for Synchronous Replication

Thank you for the comments. First of all your patch does not conform with the project style. Please have a look at how other files are indented, in particular regarding brace position and ereport() arguments (any function arguments really, but in ereport your problems are more visible). Also,

Re: [HACKERS] HOOKS for Synchronous Replication

First of all your patch does not conform with the project style. Please have a look at how other files are indented, in particular regarding brace position and ereport() arguments (any function arguments really, but in ereport your problems are more visible). Also, always use ereport() for

[HACKERS] HOOKS for Synchronous Replication

Hi, The implementation of a set of hooks for efficient synchronous replication without extensive patching of Postgresql source is now available at: http://gorda.di.uminho.pt/community/ Note that this is far from a full replication product. It is made available for early feedback by

Re: [HACKERS] HOOKS for Synchronous Replication

On Mon, Jun 20, 2005 at 11:03:45AM +0100, Alfranio Correia Junior wrote: Alfranio, The implementation of a set of hooks for efficient synchronous replication without extensive patching of Postgresql source is now available at: http://gorda.di.uminho.pt/community/ Some random

Re: [HACKERS] HOOKS for Synchronous Replication

On Mon, 20 Jun 2005, Alvaro Herrera wrote: On Mon, Jun 20, 2005 at 11:03:45AM +0100, Alfranio Correia Junior wrote: Alfranio, The implementation of a set of hooks for efficient synchronous replication without extensive patching of Postgresql source is now available at: