Re: [HACKERS] Freeze avoidance of very large table.

2015-08-18 Thread Masahiko Sawada
for visibility map needs to be in core to execute regression test at least. Regards, -- Masahiko Sawada diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c index 22c5f7a..b1b6a06 100644 --- a/contrib/pgstattuple/pgstatapprox.c +++ b/contrib/pgstattuple/pgstatapprox.c @@ -87,7

Re: [HACKERS] Freeze avoidance of very large table.

2015-08-19 Thread Masahiko Sawada
On Wed, Aug 19, 2015 at 1:28 AM, Robert Haas robertmh...@gmail.com wrote: On Tue, Aug 18, 2015 at 7:27 AM, Masahiko Sawada sawada.m...@gmail.com wrote: I have encountered the much cases where pg_stat_statement, pgstattuples are required in production, so I basically agree with moving

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-08-03 Thread Masahiko Sawada
language, the syntax checker for that language is needed, via SQL or something. Otherwise we will not be able to know whether the parsing that value will be done correctly, until reloading or restarting server. Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers

Re: [HACKERS] Freeze avoidance of very large table.

2015-07-29 Thread Masahiko Sawada
VISIBILITY_MAP_FROZEN_BIT_CAT_VER in pg_upgrade.h Please review it. Regards, -- Masahiko Sawada diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c index 22c5f7a..b1b6a06 100644 --- a/contrib/pgstattuple/pgstatapprox.c +++ b/contrib/pgstattuple/pgstatapprox.c

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-10 Thread Masahiko Sawada
On Tue, Aug 11, 2015 at 1:50 AM, Alvaro Herrera alvhe...@2ndquadrant.com wrote: Masahiko Sawada wrote: This topic may have been already discussed but, why don't we use just total scanned pages and total pages? Because those numbers don't extrapolate nicely. If the density of dead tuples

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-10 Thread Masahiko Sawada
to trace these processing, and these might be changed in the future. But basically, we can trace total scanned pages of target relation easily, and such information would be enough at many case. Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-10 Thread Masahiko Sawada
of scanning single page if table doesn't have index. So if vacuum progress checker reports the only current phase information, we would not be able to know where we are in now. Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-10-21 Thread Masahiko Sawada
On Tue, Oct 20, 2015 at 8:10 PM, Beena Emerson <memissemer...@gmail.com> wrote: > > On Mon, Oct 19, 2015 at 8:47 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> >> >> Hi, >> >> Attached patch is a rough patch which supports multi sy

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-29 Thread Masahiko Sawada
On Wed, Oct 28, 2015 at 12:58 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Sat, Oct 24, 2015 at 2:24 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> >> On Sat, Oct 24, 2015 at 10:59 AM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: &g

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-29 Thread Masahiko Sawada
On Fri, Oct 30, 2015 at 1:26 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Wed, Oct 28, 2015 at 12:58 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: >> On Sat, Oct 24, 2015 at 2:24 PM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >>>

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-24 Thread Masahiko Sawada
On Sat, Oct 24, 2015 at 10:59 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Oct 5, 2015 at 9:53 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> >> On Mon, Oct 5, 2015 at 11:03 PM, Fujii Masao <masao.fu...@gmail.com> >> wrote: >

Re: [HACKERS] Freezing without cleanup lock

2015-10-26 Thread Masahiko Sawada
e useful in setting bits there. Yeah, when the all pages have frozen totally (that is, when scanned_page + vm_skipped_frozen_page equals relpages), relfrozenxid will be updated. But the item pointers of page that are frozen by such logic are never pruned. Regards, -- Masahiko Sawada -- Sent

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-22 Thread Masahiko Sawada
On Thu, Oct 22, 2015 at 4:11 PM, Torsten Zühlsdorff <mailingli...@toco-domains.de> wrote: > On 21.10.2015 02:05, Masahiko Sawada wrote: >> >> On Sat, Oct 10, 2015 at 4:20 AM, Robert Haas <robertmh...@gmail.com> >> wrote: >>> >>> On Thu, Oct 8,

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-11-12 Thread Masahiko Sawada
On Thu, Oct 29, 2015 at 11:16 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Thu, Oct 22, 2015 at 12:47 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Tue, Oct 20, 2015 at 8:10 PM, Beena Emerson <memissemer...@gmail.com> >> wrote: >

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-11-16 Thread Masahiko Sawada
On Fri, Nov 13, 2015 at 12:52 PM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, > > At Fri, 13 Nov 2015 09:07:01 +0900, Masahiko Sawada <sawada.m...@gmail.com> > wrote in <cad21aoc9vi8wogtxio3z1nwovfxbjpnftt7+5jadvhkn17u...@mail.gmail.com> >&

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-03 Thread Masahiko Sawada
another bit) as confusing. In-fact if you find "visibility and freeze > map", > as excessively long, then we can change it to "page info map" or "page state > map" now as well. > In that case, file suffix would be "_pim" or "_psm"?

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-05 Thread Masahiko Sawada
On Wed, Nov 4, 2015 at 12:19 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Nov 4, 2015 at 4:45 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> >> On Tue, Nov 3, 2015 at 12:33 PM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: >&g

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-10-19 Thread Masahiko Sawada
On Wed, Oct 14, 2015 at 3:16 AM, Josh Berkus <j...@agliodbs.com> wrote: > On 10/13/2015 11:02 AM, Masahiko Sawada wrote: >> I thought that this feature for postgresql should be simple at first >> implementation. >> It would be good even if there are some restriction s

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-10-14 Thread Masahiko Sawada
s means that we have choose dedicated language approach instead of JSON format approach. If we want to set multi sync replication more complexly, we would not have no choice other than improvement of dedicated language. Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-08 Thread Masahiko Sawada
On Thu, Oct 8, 2015 at 7:03 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Mon, Oct 5, 2015 at 7:31 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: >> On Sat, Oct 3, 2015 at 3:41 AM, Robert Haas <robertmh...@gmail.com> wrote: >>> On Fri, Oct 2, 2015 at

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-20 Thread Masahiko Sawada
greed. > >> We could additionally use the opportunity to as a metapage, but that seems >> like an independent thing. > > I agree with that, too. > Attached the updated v18 patch fixes some bugs. Please review the patch. Regards, -- Masahiko Sawada diff --git a

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-10-13 Thread Masahiko Sawada
On Wed, Oct 14, 2015 at 3:16 AM, Josh Berkus <j...@agliodbs.com> wrote: > On 10/13/2015 11:02 AM, Masahiko Sawada wrote: >> I thought that this feature for postgresql should be simple at first >> implementation. >> It would be good even if there are some restriction s

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-10-13 Thread Masahiko Sawada
s_r_method, for example 'complex', 'json'. Though? Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-17 Thread Masahiko Sawada
ther rows on > that page, as the bit will be set back to 0? In this case, the corresponding VM both bits are cleared. Such behaviour is almost same as what postgresql is doing today. Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-17 Thread Masahiko Sawada
> OTOH being visible doesn't mean page is frozen. I understand that frozen is >> related to visibility, but still it is a separate state of page and used for >> different >> purpose. I think this is a subjective point and we could go either way, it >> is >> just a matter in which way more people are comfortable. > > I'm stickin' with what I said before, and what I think Andres is > saying too: renaming the map is a horrible idea. It produces lots of > code churn for no real benefit. We usually avoid such changes, and I > think we should do so here, too. I understood. I'm going to turn the patch back to visibility map, and just add the logic of enhancement of VACUUM FREEZE. If we want to add the other status not related to visibility into visibility map in the future, it would be worth to consider. Regards, -- Masahiko Sawada -- Regards, -- Masahiko Sawada

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-11-17 Thread Masahiko Sawada
On Tue, Nov 17, 2015 at 9:57 AM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, > > At Tue, 17 Nov 2015 01:09:57 +0900, Masahiko Sawada <sawada.m...@gmail.com> > wrote in

Re: [HACKERS] Freeze avoidance of very large table.

2015-08-26 Thread Masahiko Sawada
by diagnostic function, and compared between these result and pg_class.relallvisible(/frozen). So the essential check process is same as previous patch. We can ensure correctness by using such procedure. Regards, -- Masahiko Sawada 000_add_frozen_bit_into_visibilitymap_v12.patch Description

Re: [HACKERS] Freeze avoidance of very large table.

2015-09-03 Thread Masahiko Sawada
On Thu, Aug 27, 2015 at 1:54 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Thu, Aug 20, 2015 at 11:46 PM, Alvaro Herrera > <alvhe...@2ndquadrant.com> wrote: >> Jim Nasby wrote: >> >>> I think things like pageinspect are very different; I really

Re: [HACKERS] FSM versus GIN pending list bloat

2015-09-03 Thread Masahiko Sawada
d without error. I think it works as we expected, but the patch has some unnecessary white space. Also the today document says that pending list is cleaned up by only vacuum, we need to document about this? Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql

Re: [HACKERS] Freeze avoidance of very large table.

2015-09-07 Thread Masahiko Sawada
On Sat, Sep 5, 2015 at 7:35 AM, Simon Riggs <si...@2ndquadrant.com> wrote: > On 3 September 2015 at 18:23, Masahiko Sawada <sawada.m...@gmail.com> wrote: > >> >> The previous patch lacks some files for regression test. >> Attached fixed v12 patch. > > >

Re: [HACKERS] Freeze avoidance of very large table.

2015-09-03 Thread Masahiko Sawada
On Fri, Sep 4, 2015 at 10:35 AM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Fri, Sep 4, 2015 at 2:23 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: >> On Thu, Aug 27, 2015 at 1:54 AM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >>> On Thu

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-09-16 Thread Masahiko Sawada
ccounts; ERROR: floating-point exception DETAIL: An invalid floating-point operation was signaled. This probably means an out-of-range result or an invalid operation, such as division by zero. STATEMENT: vacuum freeze pgbench_accounts ; TRAP: FailedAssertion("!((beentry->st_changecount & 1)

Re: [HACKERS] Summary of plans to avoid the annoyance of Freezing

2015-09-15 Thread Masahiko Sawada
pgrading to 9.6. The visibility map is not critical information, and is generated by VACUUM. So we can drop it and create new visibility map by doing VACUUM, if table size is not large. Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-02 Thread Masahiko Sawada
t's worth renaming only the file extension while > there are many places where "visibility map" and "vm" are used, > for example, log messages, function names, variables, etc. > > I'd be inclined to keep calling it the visibility map (vm) even if it > also contains fre

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-02 Thread Masahiko Sawada
On Sat, Oct 3, 2015 at 12:23 AM, Alvaro Herrera <alvhe...@2ndquadrant.com> wrote: > Masahiko Sawada wrote: > Thank you for taking time to review this feature. Attached the latest version patch (v15). >> @@ -2972,10 +2981,15 @@ l1: >>*/ >>

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-09-28 Thread Masahiko Sawada
On Mon, Sep 28, 2015 at 11:03 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Sep 24, 2015 at 8:37 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> When we run "VACUUM;", the all tables of current database will be vacuumed. >> So pg_st

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-09-24 Thread Masahiko Sawada
bove as well, with something like that I imagine: > param = 'value1,' \ > 'value2,' \ > 'value3' > and this reads as 'value1,value2,value3'. This would benefit as well > for other parameters. > I agree with adding support for multi-line GUC parameters. But I though it is: p

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-09-24 Thread Masahiko Sawada
splayed. Could you check this on your environment? Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Freeze avoidance of very large table.

2015-09-18 Thread Masahiko Sawada
On Fri, Sep 18, 2015 at 6:13 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Fri, Sep 4, 2015 at 2:55 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: >> On Fri, Sep 4, 2015 at 10:35 AM, Fujii Masao <masao.fu...@gmail.com> wrote: >>> On Fri, Sep 4, 2015 a

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-05 Thread Masahiko Sawada
be problems after wraparound. > Thank you for your advice. I understood. I changed the patch so that PageClearAllVisible clear both bits, and removed ClearAllFrozen. Attached the latest v16 patch which contains draft version documentation patch. Regards, -- Masahiko Sawada diff --git a/contrib

Re: [HACKERS] Freeze avoidance of very large table.

2015-10-05 Thread Masahiko Sawada
On Mon, Oct 5, 2015 at 11:03 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Fri, Oct 2, 2015 at 8:14 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: >>> +#define Anum_pg_class_relallfrozen12 >>> Why is pg_class.relallfrozen necessary? ISTM tha

[HACKERS] Tab-comletion for RLS

2015-12-08 Thread Masahiko Sawada
ter branch and REL9_5_STABLE.​ * 001_fix_tab_completion_rls_for_95.patch fixes #3 problem, and is for only REL9_5_STABLE. ​Regards,​ -- Masahiko Sawada 000_fix_tab_completion_rls.patch Description: Binary data 001_fix_tab_completion_rls_for_95.patch Description: Binary data -- Sent via pgsql-hackers ma

Re: [HACKERS] Freeze avoidance of very large table.

2015-12-04 Thread Masahiko Sawada
On Fri, Dec 4, 2015 at 9:51 PM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Tue, Dec 1, 2015 at 10:40 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Tue, Dec 1, 2015 at 3:04 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: >>> On Mon, Nov 30, 201

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-12-09 Thread Masahiko Sawada
On Wed, Nov 18, 2015 at 2:06 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Tue, Nov 17, 2015 at 7:52 PM, Kyotaro HORIGUCHI > <horiguchi.kyot...@lab.ntt.co.jp> wrote: >> Oops. >> >> At Tue, 17 Nov 2015 19:40:10 +0900 (Tokyo Standard Time), K

Re: [HACKERS] Freeze avoidance of very large table.

2015-12-01 Thread Masahiko Sawada
On Tue, Dec 1, 2015 at 3:04 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Mon, Nov 30, 2015 at 9:18 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: >>> On Tue, Nov 24, 201

Re: [HACKERS] Freeze avoidance of very large table.

2015-12-02 Thread Masahiko Sawada
I > recommend to define the frozenpages as BlockNumber. > PgStat_MsgVacuum has a corresponding member defined as int32. I agree with you. Fixed. > pg_upgrade.c: > > BITS_PER_HEAPBLOCK is defined in two c files with the same > definition. This might be better to

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-12-11 Thread Masahiko Sawada
On Wed, Dec 9, 2015 at 8:59 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Wed, Nov 18, 2015 at 2:06 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Tue, Nov 17, 2015 at 7:52 PM, Kyotaro HORIGUCHI >> <horiguchi.kyot...@lab.ntt.co.jp> wro

Re: [HACKERS] Tab-comletion for RLS

2015-12-11 Thread Masahiko Sawada
On Thu, Dec 10, 2015 at 11:07 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Dec 8, 2015 at 8:32 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: >> I found some lacks of tab-completion for RLS in 9.5. >> >> * ALTER POLICY [TAB] >> I exp

Re: [HACKERS] Freeze avoidance of very large table.

2015-12-17 Thread Masahiko Sawada
, 2015 at 10:48:04PM +0530, Masahiko Sawada wrote: >>>> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: >>>> > On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada <sawada.m...@gmail.com> >>>> > wrote: >>>&g

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-12-17 Thread Masahiko Sawada
On Mon, Dec 14, 2015 at 2:57 PM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > Thank you for the new patch. > > At Wed, 9 Dec 2015 20:59:20 +0530, Masahiko Sawada <sawada.m...@gmail.com> > wrote in <cad21aodcn1ftoccyrqpu6fmy1xnpddakdtcbhw1r9m1mpm0...@mai

[HACKERS] comment typo in RewindTest.pm

2016-01-04 Thread Masahiko Sawada
Hi all, Attached patch fixes the comment typo in RewindTest.pm Please find it. Regards, -- Masahiko Sawada diff --git a/src/bin/pg_rewind/RewindTest.pm b/src/bin/pg_rewind/RewindTest.pm index c1c7d1f..41d294e 100644 --- a/src/bin/pg_rewind/RewindTest.pm +++ b/src/bin/pg_rewind/RewindTest.pm

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-01-03 Thread Masahiko Sawada
On Fri, Dec 25, 2015 at 7:21 AM, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > On Fri, Dec 25, 2015 at 8:50 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Wed, Dec 23, 2015 at 8:45 AM, Thomas Munro >> <thomas.mu...@enterprisedb.com> wrote:

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-12-24 Thread Masahiko Sawada
On Wed, Dec 23, 2015 at 8:45 AM, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > On Wed, Dec 23, 2015 at 3:50 PM, Thomas Munro > <thomas.mu...@enterprisedb.com> wrote: >> On Fri, Dec 18, 2015 at 7:38 AM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote

[HACKERS] ALTER ROLE SET/RESET for multiple options

2015-12-24 Thread Masahiko Sawada
)=# SELECT * FROM pg_db_role_setting ; setdatabase | setrole |setconfig -+-+- 0 | 16384 | {application_name=HOGE} (1 row) Attached draft v1 patch. Please give me feedback. Regards, -- Masahiko Sawada 000_alter_role_multiple_set_v1

Re: [HACKERS] Freeze avoidance of very large table.

2015-12-28 Thread Masahiko Sawada
. > > Maybe you are trying to say something like "only those pages which > require freezing are frozen?". > I was thinking the same as what Horiguchi-san said. That is, even if relfrozenxid is old enough, freezing on the whole table is not required if the table are marked as "

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-24 Thread Masahiko Sawada
On Mon, Nov 23, 2015 at 6:27 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Sun, Nov 22, 2015 at 8:16 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: > >> Thank you for taking the time to review this patch! >> The updated version patch is attached. >

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-30 Thread Masahiko Sawada
On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> >> Yeah, we need to consider to compute checksum if enabled. >> I've changed the patch, and attac

Re: [HACKERS] Support for N synchronous standby servers - take 2

2015-11-18 Thread Masahiko Sawada
yot...@lab.ntt.co.jp> >> Hello, >> >> At Tue, 17 Nov 2015 18:13:11 +0900, Masahiko Sawada <sawada.m...@gmail.com> >> wrote in

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-22 Thread Masahiko Sawada
On Sat, Nov 21, 2015 at 6:50 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: > On Thu, Nov 19, 2015 at 6:44 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Thu, Nov 19, 2015 at 5:54 AM, Jeff Janes <jeff.ja...@gmail.com> wrote: >>> On Wed, Nov 18

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-19 Thread Masahiko Sawada
can arrange to > link everything else and rewrite just the _vm, which is a tiny portion > of the data directory. I don't think that -k promises to link > everything it possibly can. I agree. I've changed the patch so that. pg_upgarde creates new _vm file and rewrites it even if upgrading to

Re: [HACKERS] Freeze avoidance of very large table.

2015-11-17 Thread Masahiko Sawada
On Tue, Nov 17, 2015 at 7:29 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > > > On Tue, Nov 17, 2015 at 10:45 AM, Robert Haas <robertmh...@gmail.com> wrote: >> On Sun, Nov 15, 2015 at 1:47 AM, Amit Kapila <amit.kapil...@gmail.com> >> wrote: >>>

Re: [HACKERS] Freeze avoidance of very large table.

2016-01-12 Thread Masahiko Sawada
On Mon, Dec 28, 2015 at 6:38 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Mon, Dec 21, 2015 at 11:54 PM, Robert Haas <robertmh...@gmail.com> wrote: >> On Mon, Dec 21, 2015 at 3:27 AM, Kyotaro HORIGUCHI >> <horiguchi.kyot...@lab.ntt.co.jp> wrote: >>

Re: [HACKERS] Reviewing freeze map code

2016-06-03 Thread Masahiko Sawada
On Sat, Jun 4, 2016 at 12:08 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Jun 3, 2016 at 10:49 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> That patch also incorporates the following review comment. >> We can push at least this fix

Re: [HACKERS] Reviewing freeze map code

2016-06-06 Thread Masahiko Sawada
ing this parameter as a reloption. We set it true only when the serious bug is discovered and we want to re-generate the visibility maps of specific tables. I thought that control by GUC parameter would be convenient rather than adding the new option. Regards, -- Masahiko Sawada -- Sent via pgsql-

Re: [HACKERS] Reviewing freeze map code

2016-06-06 Thread Masahiko Sawada
On Sat, Jun 4, 2016 at 1:46 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Sat, Jun 4, 2016 at 12:59 AM, Robert Haas <robertmh...@gmail.com> wrote: >> On Fri, Jun 3, 2016 at 11:21 AM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >>>>

Re: [HACKERS] Reviewing freeze map code

2016-06-08 Thread Masahiko Sawada
On Wed, Jun 8, 2016 at 12:19 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Jun 7, 2016 at 10:10 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> >> Thank you for implementing the patch. >> >> I've not test it deeply but here are som

[HACKERS] Improve tab completion for USER MAPPING

2016-06-06 Thread Masahiko Sawada
] hoge_user masahiko pg_signal_backend After manual input of the 'mapping', 'for' is added by tab completion. It means that the tab completion for 'mapping' is not working. Patch attached. Please review it. Regards, -- Masahiko Sawada fix_tab_completion_for_user_mapping.patch

Re: [HACKERS] Reviewing freeze map code

2016-06-07 Thread Masahiko Sawada
le. That is, it doesn't care in case where the all-frozen page mistakenly has dead-frozen tuple. I think this tool should check such case, otherwise the function name would need to be changed. + /* Clean up. */ + if (vmbuffer != InvalidBuffer) + ReleaseBuffer(vmbuffer); I t

Re: [HACKERS] Reviewing freeze map code

2016-06-10 Thread Masahiko Sawada
ally made the some situations where database is corrupted and tested it, but ISTM that this tool works fine. It doesn't mean proposing as a new feature of course, but please use it as appropriate. Regards, -- Masahiko Sawada corrupted_test.sql Description: Binary data set_spurious_vm_status.patch

Re: [HACKERS] Reviewing freeze map code

2016-06-03 Thread Masahiko Sawada
new_cur += BITS_PER_HEAPBLOCK; >> >> I'm not sure I'm understanding the point of the BITS_PER_HEAPBLOCK_OLD >> stuff - as long as it's hardcoded into rewriteVisibilityMap() we'll not >> be able to have differing ones anyway, should we decide to add a

Re: [HACKERS] Reviewing freeze map code

2016-06-03 Thread Masahiko Sawada
On Sat, Jun 4, 2016 at 12:41 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Jun 3, 2016 at 10:25 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >>>> + charnew_vmbuf[BLCKSZ]; >>>> +

Re: [HACKERS] Reviewing freeze map code

2016-06-03 Thread Masahiko Sawada
On Sat, Jun 4, 2016 at 12:59 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Jun 3, 2016 at 11:21 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >>> Can you submit that part as a separate patch? >> >> Attached. > > Thanks, committed. >

Re: [HACKERS] Reviewing freeze map code

2016-06-15 Thread Masahiko Sawada
nd errors out. Backend #2 writes the page to the OS. DBA > takes a hot backup, tearing the page in the middle of XMAX. Oops. > > I'm not sure what to do about this: this part of the heap_update() > logic has been like this forever, and I assume that if it were easy to > refactor this away, somebo

Re: [HACKERS] Reviewing freeze map code

2016-06-03 Thread Masahiko Sawada
On Fri, Jun 3, 2016 at 11:03 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Jun 2, 2016 at 11:24 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> Attached patch optimises skipping pages logic so that blkno can jump to >> next_unskippable_block dire

Re: [HACKERS] Reviewing freeze map code

2016-06-02 Thread Masahiko Sawada
> Yep, I had the same problem. >> Hm. This also doubles the number of VM accesses. While I guess that's >> not noticeable most of the time, it's still not nice; especially when a >> large relation is entirely frozen, because it'll mean we'll sequentially >> go through the

Re: [HACKERS] Reviewing freeze map code

2016-06-01 Thread Masahiko Sawada
/* >> +* If the page is marked as all-visible but not all-frozen, >> we should >> +* so mark it. Note that all_frozen is only valid if >> all_visible is >> +* true, so we must check both. >> +

Re: [HACKERS] Rename synchronous_standby_names?

2016-06-01 Thread Masahiko Sawada
;>>> synchronous_standbys (adjust to correct english if necesary)? >>> >>> >>> If the existing values are still going to be accepted, then I would leave >>> it as is. >> >> >> +1 > > +1. We've made quite a lot of deal to take a

Re: [HACKERS] Reviewing freeze map code

2016-05-29 Thread Masahiko Sawada
your > efforts toward speedy resolution. Thanks. > > [1] > http://www.postgresql.org/message-id/20160527025039.ga447...@tornado.leadboat.com Thank you for notification. Regarding check tool for visibility map is still under the discussion. I'm going to address other review comments,

[HACKERS] Actuall row count of Parallel Seq Scan in EXPLAIN ANALYZE .

2016-06-20 Thread Masahiko Sawada
(1000 maybe better) After spent time to investigate this behaviour, ISTM that the problem is nloops of Parallel Seq Scan. Parallel Seq Scan is done only once, but nloops is incremented to 3. So its "actual rows" is calculated 333(1000 / 3) at explain.c:L1223. Is it a bug? Re

Re: [HACKERS] Actuall row count of Parallel Seq Scan in EXPLAIN ANALYZE .

2016-06-20 Thread Masahiko Sawada
On Mon, Jun 20, 2016 at 3:42 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Mon, Jun 20, 2016 at 11:48 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> >> Hi all, >> >> My colleague noticed that the output of EXPLAIN ANALYZE doesn't work >

[HACKERS] Comment and function argument names are mismatched in bugmgr.c.

2016-06-23 Thread Masahiko Sawada
kContextInit(WritebackContext *context, int *max_coalesce); Regards, -- Masahiko Sawada diff --git a/src/backend/storage/buffer/bufmgr.c b/src/backend/storage/buffer/bufmgr.c index a67e518..76ade37 100644 --- a/src/backend/storage/buffer/bufmgr.c +++ b/src/backend/storage/buffer/bufmgr.c @@ -4160

Re: [HACKERS] forcing a rebuild of the visibility map

2016-06-16 Thread Masahiko Sawada
On Thu, Jun 16, 2016 at 10:03 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Jun 16, 2016 at 2:33 AM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> Option name DISABLE_PAGE_SKIPPING is good to me. >> I'm still working on this, but

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-01-18 Thread Masahiko Sawada
On Mon, Jan 18, 2016 at 1:20 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sun, Jan 17, 2016 at 11:09 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Wed, Jan 13, 2016 at 1:54 AM, Alvaro Herrera wrote: >> * Confirm value of pg_stat_repli

Re: [HACKERS] Freeze avoidance of very large table.

2016-01-17 Thread Masahiko Sawada
On Wed, Jan 13, 2016 at 12:16 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Mon, Dec 28, 2015 at 6:38 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Mon, Dec 21, 2015 at 11:54 PM, Robert Haas <robertmh...@gmail.com> wrote: >>> O

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-01-17 Thread Masahiko Sawada
On Wed, Jan 13, 2016 at 1:54 AM, Alvaro Herrera wrote: > Michael Paquier wrote: >> On Fri, Jan 8, 2016 at 1:53 PM, Kyotaro HORIGUCHI >> wrote: >> > Hello, >> > >> > At Mon, 4 Jan 2016 15:29:34 +0900, Michael Paquier >> >

Re: [HACKERS] Reviewing freeze map code

2016-06-28 Thread Masahiko Sawada
or out - and there's certainly callers > doing that - we'd "leak" a toasted datum. Sorry for interrupt you, but I have a question about this case. Is there case where we back to l2 after we created toasted datum(called toast_insert_or_update)? IIUC, after we stored toast datum we just insert heap tuple and log WAL (or error out for some reasons). Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] forcing a rebuild of the visibility map

2016-06-16 Thread Masahiko Sawada
ew comments. --- +CREATE FUNCTION pg_truncate_visibility_map(regclass) +RETURNS void +AS 'MODULE_PATHNAME', 'pg_truncate_visibility_map' +LANGUAGE C STRICT +PARALLEL UNSAFE; -- let's not make this any more dangerous + "REVOKE ALL ON FUNCTION pg_truncate_visibility_map(regclass) FROM PUBLIC;&quo

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-02-09 Thread Masahiko Sawada
LSNs then this function return false. * SyncRepGetSyncStandbysFn : This function obtains array of walsnd positions of its standby members that are considered as synchronous. This implementation might not good in some reason, so please give me feedbacks. And I will create new commitfest entry fo

Re: [HACKERS] Freeze avoidance of very large table.

2016-02-11 Thread Masahiko Sawada
Thank you for reviewing this patch! On Wed, Feb 10, 2016 at 4:39 PM, Kyotaro HORIGUCHI <horiguchi.kyot...@lab.ntt.co.jp> wrote: > Hello, > > At Thu, 4 Feb 2016 02:32:29 +0900, Masahiko Sawada <sawada.m...@gmail.com> > wrote in <cad21aob1hnz7thwyjqkve78gq5+pyedbbkjapb

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-02-09 Thread Masahiko Sawada
On Wed, Feb 10, 2016 at 9:18 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Feb 10, 2016 at 2:57 AM, Fujii Masao <masao.fu...@gmail.com> wrote: >> On Wed, Feb 10, 2016 at 1:36 AM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >>>

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-02-05 Thread Masahiko Sawada
0 | 2 | potential || 2 | | | london3 ||0 | 3 | potential | | 2 | |

Re: [HACKERS] Existence check for suitable index in advance when concurrently refreshing.

2016-01-27 Thread Masahiko Sawada
On Wed, Jan 27, 2016 at 4:42 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Tue, Jan 26, 2016 at 9:33 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> Hi all, >> >> In concurrently refreshing materialized view, we check whether that >> m

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-01-28 Thread Masahiko Sawada
On Thu, Jan 28, 2016 at 8:05 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Wed, Jan 20, 2016 at 2:35 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Tue, Jan 19, 2016 at 1:52 AM, Thom Brown <t...@linux.com> wrote: >>> On 3 January 2

Re: [HACKERS] Freeze avoidance of very large table.

2016-02-01 Thread Masahiko Sawada
On Tue, Feb 2, 2016 at 10:15 AM, Jim Nasby <jim.na...@bluetreble.com> wrote: > On 2/1/16 4:59 PM, Alvaro Herrera wrote: >> >> Masahiko Sawada wrote: >> >>> Attached updated version patch. >>> Please review it. >> >> >> In pg

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-02-01 Thread Masahiko Sawada
On Sun, Jan 31, 2016 at 8:58 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sun, Jan 31, 2016 at 5:28 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Sun, Jan 31, 2016 at 5:18 PM, Michael Paquier >> <michael.paqu...@gmail.com> wrote:

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-01-31 Thread Masahiko Sawada
On Sun, Jan 31, 2016 at 1:17 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Thu, Jan 28, 2016 at 10:10 PM, Masahiko Sawada wrote: >> By the discussions so far, I'm planning to have several replication >> methods such as 'quorum', 'complex' in the feature, and th

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-01-31 Thread Masahiko Sawada
On Sun, Jan 31, 2016 at 5:18 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sun, Jan 31, 2016 at 5:08 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Sun, Jan 31, 2016 at 1:17 PM, Michael Paquier >> <michael.paqu...@gmail.com> wrote:

Re: [HACKERS] Freeze avoidance of very large table.

2016-02-02 Thread Masahiko Sawada
On Tue, Feb 2, 2016 at 11:42 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Tue, Feb 2, 2016 at 10:15 AM, Jim Nasby <jim.na...@bluetreble.com> wrote: >> On 2/1/16 4:59 PM, Alvaro Herrera wrote: >>> >>> Masahiko Sawada wrote: >>> >>

Re: [HACKERS] Support for N synchronous standby servers - take 2

2016-02-02 Thread Masahiko Sawada
On Mon, Feb 1, 2016 at 11:28 PM, Fujii Masao <masao.fu...@gmail.com> wrote: > On Mon, Feb 1, 2016 at 5:36 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: >> On Sun, Jan 31, 2016 at 8:58 PM, Michael Paquier >> <michael.paqu...@gmail.com> wrote: >>>

Re: [HACKERS] Freeze avoidance of very large table.

2016-02-02 Thread Masahiko Sawada
ed again in the future, we could add other libraries for convert Regards, -- Masahiko Sawada -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

  1   2   3   4   5   6   7   8   >