Re: [HACKERS] Freeze avoidance of very large table.

On 03/11/2016 09:48 AM, Masahiko Sawada wrote: Thank you so much! What I wanted deal with in thread is almost done. I'm going to more test the feature for 9.6 releasing. Nicely done! Regards, -- Masahiko Sawada -- Command Prompt, Inc. http://the.postgres.company/

Re: [HACKERS] Freeze avoidance of very large table.

On Sat, Mar 12, 2016 at 2:37 AM, Robert Haas wrote: > On Thu, Mar 10, 2016 at 10:47 PM, Masahiko Sawada > wrote: >>> Thanks. I adopted some of your suggested, rejected others, fixed a >>> few minor things that I missed previously, and committed

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Mar 10, 2016 at 10:47 PM, Masahiko Sawada wrote: >> Thanks. I adopted some of your suggested, rejected others, fixed a >> few minor things that I missed previously, and committed this. If you >> think any of the changes that I rejected still have merit, please >>

Re: [HACKERS] Freeze avoidance of very large table.

On Fri, Mar 11, 2016 at 6:16 AM, Robert Haas wrote: > On Thu, Mar 10, 2016 at 1:41 PM, Masahiko Sawada > wrote: >> On Fri, Mar 11, 2016 at 1:03 AM, Robert Haas wrote: >>> This 001 patch looks so little like what I was

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Mar 10, 2016 at 1:41 PM, Masahiko Sawada wrote: > On Fri, Mar 11, 2016 at 1:03 AM, Robert Haas wrote: >> This 001 patch looks so little like what I was expecting that I >> decided to start over from scratch. The new version I wrote is >>

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Mar 10, 2016 at 1:41 PM, Masahiko Sawada wrote: > On Fri, Mar 11, 2016 at 1:03 AM, Robert Haas wrote: >> This 001 patch looks so little like what I was expecting that I >> decided to start over from scratch. The new version I wrote is >>

Re: [HACKERS] Freeze avoidance of very large table.

On Fri, Mar 11, 2016 at 1:03 AM, Robert Haas wrote: > This 001 patch looks so little like what I was expecting that I > decided to start over from scratch. The new version I wrote is > attached here. I don't understand why your version tinkers with the > logic for setting

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Mar 10, 2016 at 8:51 AM, Masahiko Sawada wrote: > After some further thought, I thought that it's better to add check > logic for result of rewriting visibility map to upgrading logic rather > than regression test in order to ensure that rewriting visibility map >

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Mar 10, 2016 at 3:27 PM, Masahiko Sawada wrote: > Thank you for reviewing! > Attached updated patch. > > > On Thu, Mar 10, 2016 at 3:37 AM, Robert Haas wrote: >> On Wed, Mar 9, 2016 at 9:09 AM, Masahiko Sawada >>

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Mar 9, 2016 at 9:09 AM, Masahiko Sawada wrote: > * 001 patch : Incorporated the documentation suggestions and updated > logic a little. This 001 patch looks so little like what I was expecting that I decided to start over from scratch. The new version I wrote is

Re: [HACKERS] Freeze avoidance of very large table.

Thank you for reviewing! Attached updated patch. On Thu, Mar 10, 2016 at 3:37 AM, Robert Haas wrote: > On Wed, Mar 9, 2016 at 9:09 AM, Masahiko Sawada > wrote: Attached latest 2 patches. >> * 000 patch : Incorporated the review comments and made

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Mar 9, 2016 at 9:09 AM, Masahiko Sawada wrote: Attached latest 2 patches. > * 000 patch : Incorporated the review comments and made rewriting > logic more clearly. That's better, thanks. But your comments don't survive pgindent. After running pgindent, I get this:

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Mar 9, 2016 at 3:38 AM, Robert Haas wrote: > On Tue, Mar 8, 2016 at 12:59 PM, Masahiko Sawada > wrote: >>> How about instead changing things so that we specifically reject >>> indexes? And maybe some kind of a check that will reject

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Mar 8, 2016 at 12:59 PM, Masahiko Sawada wrote: >> How about instead changing things so that we specifically reject >> indexes? And maybe some kind of a check that will reject anything >> that lacks a relfilnode? That seems like it would be more on point. > > I

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Mar 8, 2016 at 12:49 PM, Tom Lane wrote: >> However, after some further thought, I think we might actually be OK. >> If a page goes from all-frozen to not-all-frozen while VACUUM is >> running, any new XID added to the page must be newer than the >> oldestXmin value

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Mar 9, 2016 at 1:23 AM, Jeff Janes wrote: > On Tue, Mar 8, 2016 at 5:30 AM, Robert Haas wrote: > I left out the relkind check from the final commit because, for one > thing, the check you added isn't actually right: toast relations can > also

Re: [HACKERS] Freeze avoidance of very large table.

Robert Haas writes: > The patch makes some attempt to update the comment mechanically, but > that's not nearly enough. That comment is explaining that you *can't* > rely on the visibility map to tell you *for sure* that a page does not > require vacuuming. For current

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Mar 7, 2016 at 12:41 PM, Masahiko Sawada wrote: > Attached latest version optimisation patch. > I'm still consider regarding pg_upgrade regression test code, so I > will submit that patch later. I just spent some time looking at this and I'm a bit worried about the

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Mar 8, 2016 at 5:30 AM, Robert Haas wrote: > On Tue, Mar 8, 2016 at 7:26 AM, Masahiko Sawada wrote: >> Regarding pg_visibility module, I'd like to share some bugs and >> propose to add a relation type condition to each functions. > > OK,

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Mar 8, 2016 at 8:30 AM, Robert Haas wrote: > On Tue, Mar 8, 2016 at 7:26 AM, Masahiko Sawada wrote: >> Regarding pg_visibility module, I'd like to share some bugs and >> propose to add a relation type condition to each functions. > > OK,

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Mar 8, 2016 at 7:26 AM, Masahiko Sawada wrote: > Regarding pg_visibility module, I'd like to share some bugs and > propose to add a relation type condition to each functions. OK, thanks. > Including it, I've attached remaining 2 patches; one is removing page >

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Mar 8, 2016 at 1:20 PM, Kyotaro HORIGUCHI wrote: > Hello, thank you for updating this tool. > > At Mon, 7 Mar 2016 14:03:08 -0500, Robert Haas wrote > in >> On

Re: [HACKERS] Freeze avoidance of very large table.

Hello, thank you for updating this tool. At Mon, 7 Mar 2016 14:03:08 -0500, Robert Haas wrote in > On Mon, Mar 7, 2016 at 12:41 PM, Masahiko Sawada > wrote: > > Attached latest

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Mar 7, 2016 at 4:50 PM, Robert Haas wrote: > Here's an updated patch with an API that I think is much more > reasonable to expose to users, and documentation! It assumes that the > patch I posted a few hours ago to remove PD_ALL_FROZEN will be > accepted; if that

Re: [HACKERS] Freeze avoidance of very large table.

On Sat, Mar 5, 2016 at 9:25 AM, Masahiko Sawada wrote: >> I actually think end-users might well want to use it. Also, I created >> it by hacking up pg_freespacemap, so it may make sense to have it in >> the same place. >> I would also be tempted to add an additional C

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Mar 7, 2016 at 12:41 PM, Masahiko Sawada wrote: > Attached latest version optimisation patch. > I'm still consider regarding pg_upgrade regression test code, so I > will submit that patch later. I was thinking more about this today and I think that we don't

Re: [HACKERS] Freeze avoidance of very large table.

On Sat, Mar 5, 2016 at 11:25 PM, Masahiko Sawada wrote: > On Sat, Mar 5, 2016 at 1:25 AM, Robert Haas wrote: >> On Wed, Mar 2, 2016 at 6:41 PM, Tom Lane wrote: >>> Jim Nasby writes: On 3/2/16 4:21

Re: [HACKERS] Freeze avoidance of very large table.

On Sat, Mar 5, 2016 at 1:25 AM, Robert Haas wrote: > On Wed, Mar 2, 2016 at 6:41 PM, Tom Lane wrote: >> Jim Nasby writes: >>> On 3/2/16 4:21 PM, Peter Geoghegan wrote: I think you should commit this. The chances of anyone

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Mar 2, 2016 at 6:41 PM, Tom Lane wrote: > Jim Nasby writes: >> On 3/2/16 4:21 PM, Peter Geoghegan wrote: >>> I think you should commit this. The chances of anyone other than you >>> and Masahiko recalling that you developed this tool in 3

Re: [HACKERS] Freeze avoidance of very large table.

At Wed, 2 Mar 2016 17:57:27 -0600, Jim Nasby wrote in <56d77de7.7080...@bluetreble.com> > On 3/2/16 5:41 PM, Tom Lane wrote: > > Jim Nasby writes: > >> On 3/2/16 4:21 PM, Peter Geoghegan wrote: > >>> I think you should commit this. The chances

Re: [HACKERS] Freeze avoidance of very large table.

On 3/2/16 5:41 PM, Tom Lane wrote: Jim Nasby writes: On 3/2/16 4:21 PM, Peter Geoghegan wrote: I think you should commit this. The chances of anyone other than you and Masahiko recalling that you developed this tool in 3 years is essentially nil. I think that the

Re: [HACKERS] Freeze avoidance of very large table.

Jim Nasby writes: > On 3/2/16 4:21 PM, Peter Geoghegan wrote: >> I think you should commit this. The chances of anyone other than you >> and Masahiko recalling that you developed this tool in 3 years is >> essentially nil. I think that the cost of committing a

Re: [HACKERS] Freeze avoidance of very large table.

On 3/2/16 4:21 PM, Peter Geoghegan wrote: I think you should commit this. The chances of anyone other than you and Masahiko recalling that you developed this tool in 3 years is essentially nil. I think that the cost of committing a developer-level debugging tool like this is very low. Modules

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Mar 1, 2016 at 6:51 PM, Robert Haas wrote: > I removed the pgstat stuff. I'm not sure we want that stuff in that > form; it doesn't seem to fit with the rest of what's in that view, and > it wasn't reliable in my testing. I did however throw together a > little

Re: [HACKERS] Freeze avoidance of very large table.

Thank you for revising and commiting this. At Tue, 1 Mar 2016 21:51:55 -0500, Robert Haas wrote in > On Thu, Feb 18, 2016 at 3:45 AM, Masahiko Sawada > wrote: > > Attached

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Feb 18, 2016 at 3:45 AM, Masahiko Sawada wrote: > Attached updated 5 patches. > I would like to explain these patch shortly again here to make > reviewing more easier. > > We can divided these patches into 2 purposes. > > 1. Freeze map > 000_ patch adds additional

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Feb 17, 2016 at 4:29 PM, Masahiko Sawada wrote: > On Wed, Feb 17, 2016 at 4:08 AM, Bruce Momjian wrote: >> On Tue, Feb 16, 2016 at 03:57:01PM -0300, Alvaro Herrera wrote: >>> Masahiko Sawada wrote: >>> > On Wed, Feb 17, 2016 at 12:02 AM, Bruce

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Feb 17, 2016 at 4:08 AM, Bruce Momjian wrote: > On Tue, Feb 16, 2016 at 03:57:01PM -0300, Alvaro Herrera wrote: >> Masahiko Sawada wrote: >> > On Wed, Feb 17, 2016 at 12:02 AM, Bruce Momjian wrote: >> > > On Tue, Feb 16, 2016 at 11:56:25PM +0900,

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Feb 16, 2016 at 03:57:01PM -0300, Alvaro Herrera wrote: > Masahiko Sawada wrote: > > On Wed, Feb 17, 2016 at 12:02 AM, Bruce Momjian wrote: > > > On Tue, Feb 16, 2016 at 11:56:25PM +0900, Masahiko Sawada wrote: > > >> > I agreed on ripping out the converter plugin

Re: [HACKERS] Freeze avoidance of very large table.

Masahiko Sawada wrote: > On Wed, Feb 17, 2016 at 12:02 AM, Bruce Momjian wrote: > > On Tue, Feb 16, 2016 at 11:56:25PM +0900, Masahiko Sawada wrote: > >> > I agreed on ripping out the converter plugin ability of pg_upgrade. > >> > Remember pg_upgrade was originally written by

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Feb 17, 2016 at 12:02 AM, Bruce Momjian wrote: > On Tue, Feb 16, 2016 at 11:56:25PM +0900, Masahiko Sawada wrote: >> > I agreed on ripping out the converter plugin ability of pg_upgrade. >> > Remember pg_upgrade was originally written by EnterpriseDB staff, and I >> >

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Feb 16, 2016 at 11:56:25PM +0900, Masahiko Sawada wrote: > > I agreed on ripping out the converter plugin ability of pg_upgrade. > > Remember pg_upgrade was originally written by EnterpriseDB staff, and I > > think they expected their closed-source fork of Postgres might need a > > custom

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Feb 16, 2016 at 6:13 AM, Bruce Momjian wrote: > On Wed, Feb 10, 2016 at 04:39:15PM +0900, Kyotaro HORIGUCHI wrote: >> > I still agree with this plugin approach, but I felt it's still >> > complicated a bit, and I'm concerned that patch size has been >> > increased. >> >

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Feb 10, 2016 at 04:39:15PM +0900, Kyotaro HORIGUCHI wrote: > > I still agree with this plugin approach, but I felt it's still > > complicated a bit, and I'm concerned that patch size has been > > increased. > > Please give me feedbacks. > > Yeah, I feel the same. What make it worse, the

Re: [HACKERS] Freeze avoidance of very large table.

On Sun, Feb 14, 2016 at 12:19 AM, Masahiko Sawada wrote: > Thank you for reviewing this patch. > I've divided 000 patch into two patches, and attached latest 4 patches in > total. Thank you! I'll go through this again as soon as I have a free moment. -- Robert Haas

Re: [HACKERS] Freeze avoidance of very large table.

Thank you for reviewing this patch! On Wed, Feb 10, 2016 at 4:39 PM, Kyotaro HORIGUCHI wrote: > Hello, > > At Thu, 4 Feb 2016 02:32:29 +0900, Masahiko Sawada > wrote in

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Feb 3, 2016 at 12:32 PM, Masahiko Sawada wrote: > I've divided the main patch into two patches; add frozen bit patch and > pg_upgrade support patch. > 000 patch is almost same as previous code. (includes small fix) > 001 patch provides rewriting visibility map as a

Re: [HACKERS] Freeze avoidance of very large table.

Hello, At Thu, 4 Feb 2016 02:32:29 +0900, Masahiko Sawada wrote in > On Tue, Feb 2, 2016 at 7:22 PM, Alvaro Herrera > wrote: > > Masahiko Sawada wrote: > >> I think we have

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Feb 2, 2016 at 7:22 PM, Alvaro Herrera wrote: > Masahiko Sawada wrote: > >> I misunderstood. Sorry for noise. >> I agree with adding conversion method as a pageConverter routine. > > \o/ > >> This patch doesn't change page layout actually, but pageConverter >>

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Feb 2, 2016 at 11:42 AM, Masahiko Sawada wrote: > On Tue, Feb 2, 2016 at 10:15 AM, Jim Nasby wrote: >> On 2/1/16 4:59 PM, Alvaro Herrera wrote: >>> >>> Masahiko Sawada wrote: >>> Attached updated version patch. Please review it.

Re: [HACKERS] Freeze avoidance of very large table.

Masahiko Sawada wrote: > I misunderstood. Sorry for noise. > I agree with adding conversion method as a pageConverter routine. \o/ > This patch doesn't change page layout actually, but pageConverter > routine checks only the page layout. > And we have to plugin named convertLayout_X_to_Y. > >

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Feb 2, 2016 at 10:05 PM, Alvaro Herrera wrote: > This patch has gotten its fair share of feedback in this fest. I moved > it to the next commitfest. Please do keep working on it and reviewers > that have additional comments are welcome. Thanks! On Tue, Feb 2,

Re: [HACKERS] Freeze avoidance of very large table.

Hello, At Tue, 2 Feb 2016 20:25:23 +0900, Masahiko Sawada wrote in

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Feb 2, 2016 at 7:22 PM, Alvaro Herrera wrote: > Masahiko Sawada wrote: > >> I misunderstood. Sorry for noise. >> I agree with adding conversion method as a pageConverter routine. > > \o/ > >> This patch doesn't change page layout actually, but pageConverter >>

Re: [HACKERS] Freeze avoidance of very large table.

This patch has gotten its fair share of feedback in this fest. I moved it to the next commitfest. Please do keep working on it and reviewers that have additional comments are welcome. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA,

Re: [HACKERS] Freeze avoidance of very large table.

Masahiko Sawada wrote: > Attached updated version patch. > Please review it. In pg_upgrade, the "page convert" functionality is there to abstract rewrites of pages being copied; your patch is circumventing it and AFAICS it makes the interface more complicated for no good reason. I think the

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Feb 2, 2016 at 10:15 AM, Jim Nasby wrote: > On 2/1/16 4:59 PM, Alvaro Herrera wrote: >> >> Masahiko Sawada wrote: >> >>> Attached updated version patch. >>> Please review it. >> >> >> In pg_upgrade, the "page convert" functionality is there to abstract >>

Re: [HACKERS] Freeze avoidance of very large table.

On 2/1/16 4:59 PM, Alvaro Herrera wrote: Masahiko Sawada wrote: Attached updated version patch. Please review it. In pg_upgrade, the "page convert" functionality is there to abstract rewrites of pages being copied; your patch is circumventing it and AFAICS it makes the interface more

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Jan 13, 2016 at 12:16 AM, Masahiko Sawada wrote: > On Mon, Dec 28, 2015 at 6:38 PM, Masahiko Sawada > wrote: >> On Mon, Dec 21, 2015 at 11:54 PM, Robert Haas wrote: >>> On Mon, Dec 21, 2015 at 3:27 AM, Kyotaro

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Dec 28, 2015 at 6:38 PM, Masahiko Sawada wrote: > On Mon, Dec 21, 2015 at 11:54 PM, Robert Haas wrote: >> On Mon, Dec 21, 2015 at 3:27 AM, Kyotaro HORIGUCHI >> wrote: >>> Hello, >>> >>> At Fri, 18 Dec 2015

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Dec 21, 2015 at 11:54 PM, Robert Haas wrote: > On Mon, Dec 21, 2015 at 3:27 AM, Kyotaro HORIGUCHI > wrote: >> Hello, >> >> At Fri, 18 Dec 2015 12:09:43 -0500, Robert Haas >> wrote in

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Dec 17, 2015 at 06:44:46AM +, Simon Riggs wrote: > >> Thank you for having a look. > > > > I would not bother mentioning this detail in the pg_upgrade manual page: > > > > +   Since the format of visibility map has been changed in version 9.6, > > +   pg_upgrade

Re: [HACKERS] Freeze avoidance of very large table.

Hello, At Fri, 18 Dec 2015 12:09:43 -0500, Robert Haas wrote in > On Thu, Dec 17, 2015 at 1:17 AM, Michael Paquier > wrote: > > I am not really getting the meaning of this

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Dec 21, 2015 at 3:27 AM, Kyotaro HORIGUCHI wrote: > Hello, > > At Fri, 18 Dec 2015 12:09:43 -0500, Robert Haas wrote > in >> On Thu, Dec 17, 2015 at 1:17 AM,

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Dec 17, 2015 at 2:26 AM, Andres Freund wrote: > On 2015-12-17 16:22:24 +0900, Michael Paquier wrote: >> On Thu, Dec 17, 2015 at 4:10 PM, Andres Freund wrote: >> > On 2015-12-17 15:56:35 +0900, Michael Paquier wrote: >> >> On Thu, Dec 17, 2015 at

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Dec 17, 2015 at 1:17 AM, Michael Paquier wrote: > I am not really getting the meaning of this sentence. Shouldn't this > be reworded something like: > "Freezing occurs on the whole table once all pages of this relation require > it." That statement isn't

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Dec 17, 2015 at 11:47 AM, Michael Paquier wrote: > On Thu, Dec 10, 2015 at 3:31 AM, Robert Haas wrote: >> On Mon, Nov 30, 2015 at 12:58 PM, Bruce Momjian wrote: >>> On Mon, Nov 30, 2015 at 10:48:04PM +0530, Masahiko

Re: [HACKERS] Freeze avoidance of very large table.

On Fri, Dec 18, 2015 at 3:17 AM, Masahiko Sawada wrote: > On Thu, Dec 17, 2015 at 11:47 AM, Michael Paquier > wrote: >> make check with pg_upgrade is failing for me: >> Visibility map rewriting test failed >> make: *** [check] Error 1 > > make

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Dec 10, 2015 at 3:31 AM, Robert Haas wrote: > On Mon, Nov 30, 2015 at 12:58 PM, Bruce Momjian wrote: >> On Mon, Nov 30, 2015 at 10:48:04PM +0530, Masahiko Sawada wrote: >>> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes wrote:

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Dec 17, 2015 at 3:44 PM, Simon Riggs wrote: > For me, rewriting the visibility map is a new data loss bug waiting to > happen. I am worried that the group is not taking seriously the potential > for catastrophe here. FWIW, I'm following this line and merging the vm

Re: [HACKERS] Freeze avoidance of very large table.

On 9 December 2015 at 18:31, Robert Haas wrote: > On Mon, Nov 30, 2015 at 12:58 PM, Bruce Momjian wrote: > > On Mon, Nov 30, 2015 at 10:48:04PM +0530, Masahiko Sawada wrote: > >> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes > wrote:

Re: [HACKERS] Freeze avoidance of very large table.

On 2015-12-17 15:56:35 +0900, Michael Paquier wrote: > On Thu, Dec 17, 2015 at 3:44 PM, Simon Riggs wrote: > > For me, rewriting the visibility map is a new data loss bug waiting to > > happen. I am worried that the group is not taking seriously the potential > > for

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Dec 17, 2015 at 4:10 PM, Andres Freund wrote: > On 2015-12-17 15:56:35 +0900, Michael Paquier wrote: >> On Thu, Dec 17, 2015 at 3:44 PM, Simon Riggs wrote: >> > For me, rewriting the visibility map is a new data loss bug waiting to >> > happen.

Re: [HACKERS] Freeze avoidance of very large table.

On 2015-12-17 16:22:24 +0900, Michael Paquier wrote: > On Thu, Dec 17, 2015 at 4:10 PM, Andres Freund wrote: > > On 2015-12-17 15:56:35 +0900, Michael Paquier wrote: > >> On Thu, Dec 17, 2015 at 3:44 PM, Simon Riggs wrote: > >> > For me, rewriting the

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Nov 30, 2015 at 12:58 PM, Bruce Momjian wrote: > On Mon, Nov 30, 2015 at 10:48:04PM +0530, Masahiko Sawada wrote: >> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes wrote: >> > On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada

Re: [HACKERS] Freeze avoidance of very large table.

On Fri, Dec 4, 2015 at 9:51 PM, Jeff Janes wrote: > On Tue, Dec 1, 2015 at 10:40 AM, Masahiko Sawada > wrote: >> On Tue, Dec 1, 2015 at 3:04 AM, Jeff Janes wrote: >>> On Mon, Nov 30, 2015 at 9:18 AM, Masahiko Sawada

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Dec 1, 2015 at 10:40 AM, Masahiko Sawada wrote: > On Tue, Dec 1, 2015 at 3:04 AM, Jeff Janes wrote: >> On Mon, Nov 30, 2015 at 9:18 AM, Masahiko Sawada >> wrote: >>> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Dec 2, 2015 at 9:30 AM, Kyotaro HORIGUCHI wrote: > Hello, > >> You're right, it's not necessary. >> Attached latest v29 patch which removes the mention in pg_upgrade >> documentation. > > The changes looks to be correct but I haven't tested. > And I have

Re: [HACKERS] Freeze avoidance of very large table.

Hello, > You're right, it's not necessary. > Attached latest v29 patch which removes the mention in pg_upgrade > documentation. The changes looks to be correct but I haven't tested. And I have some additional random comments. visibilitymap.c: In visibilitymap_set, the followint lines.

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Dec 1, 2015 at 3:04 AM, Jeff Janes wrote: > On Mon, Nov 30, 2015 at 9:18 AM, Masahiko Sawada > wrote: >> On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes wrote: >>> On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Nov 30, 2015 at 9:18 AM, Masahiko Sawada wrote: > On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes wrote: >> On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada >> wrote: >>> >>> Yeah, we need to consider to compute checksum

Re: [HACKERS] Freeze avoidance of very large table.

On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes wrote: > On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada > wrote: >> >> Yeah, we need to consider to compute checksum if enabled. >> I've changed the patch, and attached. >> Please review it. > > Thanks

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Nov 30, 2015 at 10:48:04PM +0530, Masahiko Sawada wrote: > On Sun, Nov 29, 2015 at 2:21 AM, Jeff Janes wrote: > > On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada > > wrote: > >> > >> Yeah, we need to consider to compute checksum if enabled.

Re: [HACKERS] Freeze avoidance of very large table.

On 2015-11-30 12:58:43 -0500, Bruce Momjian wrote: > I would not bother mentioning this detail in the pg_upgrade manual page: > > + Since the format of visibility map has been changed in version 9.6, > + pg_upgrade creates and rewrite new > '_vm' > + file even if upgrading from 9.5 or

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Nov 30, 2015 at 07:05:21PM +0100, Andres Freund wrote: > On 2015-11-30 12:58:43 -0500, Bruce Momjian wrote: > > I would not bother mentioning this detail in the pg_upgrade manual page: > > > > + Since the format of visibility map has been changed in version 9.6, > > + pg_upgrade

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Nov 24, 2015 at 3:13 PM, Masahiko Sawada wrote: > > Yeah, we need to consider to compute checksum if enabled. > I've changed the patch, and attached. > Please review it. Thanks for the update. This now conflicts with the updates doesn to fix pg_upgrade

Re: [HACKERS] Freeze avoidance of very large table.

On Mon, Nov 23, 2015 at 6:27 AM, Jeff Janes wrote: > On Sun, Nov 22, 2015 at 8:16 AM, Masahiko Sawada > wrote: > >> Thank you for taking the time to review this patch! >> The updated version patch is attached. > > I am skeptical about just copying

Re: [HACKERS] Freeze avoidance of very large table.

On Sat, Nov 21, 2015 at 6:50 AM, Jeff Janes wrote: > On Thu, Nov 19, 2015 at 6:44 AM, Masahiko Sawada > wrote: >> On Thu, Nov 19, 2015 at 5:54 AM, Jeff Janes wrote: >>> On Wed, Nov 18, 2015 at 11:18 AM, Jeff Janes

Re: [HACKERS] Freeze avoidance of very large table.

On Sun, Nov 22, 2015 at 8:16 AM, Masahiko Sawada wrote: > Thank you for taking the time to review this patch! > The updated version patch is attached. I am skeptical about just copying the old page header to be two new page headers. I don't know what the implications for

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Nov 19, 2015 at 6:44 AM, Masahiko Sawada wrote: > On Thu, Nov 19, 2015 at 5:54 AM, Jeff Janes wrote: >> On Wed, Nov 18, 2015 at 11:18 AM, Jeff Janes wrote: >>> >>> I get an error when running pg_upgrade from 9.4 to

Re: [HACKERS] Freeze avoidance of very large table.

On Thu, Nov 19, 2015 at 5:54 AM, Jeff Janes wrote: > On Wed, Nov 18, 2015 at 11:18 AM, Jeff Janes wrote: >> >> I get an error when running pg_upgrade from 9.4 to 9.6-this >> >> error while copying relation "mediawiki.archive" >>

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Nov 17, 2015 at 10:32 PM, Masahiko Sawada wrote: > > Attached latest v24 patch. > I've changed patch so that just adding frozen bit into visibility map. > So the size of patch is almost half of previous one. > Should there be an Assert in visibilitymap_get_status

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Nov 18, 2015 at 11:18 AM, Jeff Janes wrote: > > I get an error when running pg_upgrade from 9.4 to 9.6-this > > error while copying relation "mediawiki.archive" > ("/tmp/data/base/16414/21043_vm" to > "/tmp/data_fm/base/16400/21043_vm"): No such file or directory

Re: [HACKERS] Freeze avoidance of very large table.

On 17 November 2015 at 10:29, Masahiko Sawada wrote: > > > On Tue, Nov 17, 2015 at 10:45 AM, Robert Haas wrote: >> On Sun, Nov 15, 2015 at 1:47 AM, Amit Kapila >> wrote: >>> On Sat, Nov 14, 2015 at 1:19 AM, Andres Freund

Re: [HACKERS] Freeze avoidance of very large table.

On 17 November 2015 at 15:43, Jim Nasby wrote: > On 11/17/15 4:41 AM, Thom Brown wrote: >> >> Could someone post a TL;DR summary of what the current plan looks >> like? I can see there is a huge amount of discussion to trawl back >> through. I can see it's something to

Re: [HACKERS] Freeze avoidance of very large table.

On 11/17/15 4:41 AM, Thom Brown wrote: Could someone post a TL;DR summary of what the current plan looks like? I can see there is a huge amount of discussion to trawl back through. I can see it's something to do with the visibility map. And does it avoid freezing very large tables like the

Re: [HACKERS] Freeze avoidance of very large table.

On Wed, Nov 18, 2015 at 12:56 AM, Thom Brown wrote: > On 17 November 2015 at 15:43, Jim Nasby wrote: >> On 11/17/15 4:41 AM, Thom Brown wrote: >>> >>> Could someone post a TL;DR summary of what the current plan looks >>> like? I can see there is a huge

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Nov 17, 2015 at 10:45 AM, Robert Haas > wrote: > On Sun, Nov 15, 2015 at 1:47 AM, Amit Kapila > wrote: >> On Sat, Nov 14, 2015 at 1:19 AM, Andres Freund > wrote: >>> On 2015-10-31

Re: [HACKERS] Freeze avoidance of very large table.

On Tue, Nov 17, 2015 at 7:29 PM, Masahiko Sawada wrote: > > > On Tue, Nov 17, 2015 at 10:45 AM, Robert Haas wrote: >> On Sun, Nov 15, 2015 at 1:47 AM, Amit Kapila >> wrote: >>> On Sat, Nov 14, 2015 at 1:19 AM, Andres Freund

Re: [HACKERS] Freeze avoidance of very large table.

On Sun, Nov 15, 2015 at 1:47 AM, Amit Kapila wrote: > On Sat, Nov 14, 2015 at 1:19 AM, Andres Freund wrote: >> On 2015-10-31 11:02:12 +0530, Amit Kapila wrote: >> > On Thu, Oct 8, 2015 at 11:05 PM, Simon Riggs >> > wrote: >> >

  1   2   3   >