Re: [HACKERS] CLUSTER command progress monitor

2017-10-02 Thread Robert Haas
On Tue, Sep 12, 2017 at 8:20 AM, Tatsuro Yamada wrote: >>> I agree that progress reporting for sort is difficult. So it only reports >>> the phase ("sorting tuples") in the current design of progress monitor of >>> cluster. >>> It doesn't report counter of sort. >>

Re: [HACKERS] CLUSTER command progress monitor

2017-10-01 Thread Daniel Gustafsson
> On 12 Sep 2017, at 14:57, Tatsuro Yamada wrote: > > On 2017/09/12 21:20, Tatsuro Yamada wrote: >> On 2017/09/11 23:38, Robert Haas wrote: >>> On Sun, Sep 10, 2017 at 10:36 PM, Tatsuro Yamada >>> wrote: Thanks for the comment.

Re: [HACKERS] CLUSTER command progress monitor

2017-09-15 Thread Jeff Janes
On Wed, Aug 30, 2017 at 7:12 PM, Tatsuro Yamada < yamada.tats...@lab.ntt.co.jp> wrote: > > The view provides the information of CLUSTER command progress details as > follows > postgres=# \d pg_stat_progress_cluster >View "pg_catalog.pg_stat_progress_cluster" >Column|

Re: [HACKERS] CLUSTER command progress monitor

2017-09-12 Thread Tatsuro Yamada
On 2017/09/12 21:20, Tatsuro Yamada wrote: On 2017/09/11 23:38, Robert Haas wrote: On Sun, Sep 10, 2017 at 10:36 PM, Tatsuro Yamada wrote: Thanks for the comment. As you know, CLUSTER command uses SEQ SCAN or INDEX SCAN as a scan method by cost estimation. In

Re: [HACKERS] CLUSTER command progress monitor

2017-09-12 Thread Tatsuro Yamada
On 2017/09/11 23:38, Robert Haas wrote: On Sun, Sep 10, 2017 at 10:36 PM, Tatsuro Yamada wrote: Thanks for the comment. As you know, CLUSTER command uses SEQ SCAN or INDEX SCAN as a scan method by cost estimation. In the case of SEQ SCAN, these two phases not

Re: [HACKERS] CLUSTER command progress monitor

2017-09-11 Thread Peter Geoghegan
On Mon, Sep 11, 2017 at 7:38 AM, Robert Haas wrote: > On Sun, Sep 10, 2017 at 10:36 PM, Tatsuro Yamada > wrote: >> Thanks for the comment. >> >> As you know, CLUSTER command uses SEQ SCAN or INDEX SCAN as a scan method by >> cost estimation.

Re: [HACKERS] CLUSTER command progress monitor

2017-09-11 Thread Robert Haas
On Sun, Sep 10, 2017 at 10:36 PM, Tatsuro Yamada wrote: > Thanks for the comment. > > As you know, CLUSTER command uses SEQ SCAN or INDEX SCAN as a scan method by > cost estimation. In the case of SEQ SCAN, these two phases not overlap. > However, in INDEX SCAN, it

Re: [HACKERS] CLUSTER command progress monitor

2017-09-10 Thread Tatsuro Yamada
On 2017/09/08 18:55, Robert Haas wrote: On Wed, Aug 30, 2017 at 10:12 PM, Tatsuro Yamada wrote: 1. scanning heap 2. sort tuples These two phases overlap, though. I believe progress reporting for sorts is really hard. In the simple case where the data fits

Re: [HACKERS] CLUSTER command progress monitor

2017-09-08 Thread Robert Haas
On Wed, Aug 30, 2017 at 10:12 PM, Tatsuro Yamada wrote: > 1. scanning heap > 2. sort tuples These two phases overlap, though. I believe progress reporting for sorts is really hard. In the simple case where the data fits in work_mem, none of the work of the sort

Re: [HACKERS] CLUSTER command progress monitor

2017-09-07 Thread Tatsuro Yamada
On 2017/09/06 16:11, Michael Paquier wrote: On Wed, Sep 6, 2017 at 3:58 PM, Tatsuro Yamada wrote: I revised the patch like this: You should avoid top-posting. I see. I didn't change the name of view (pg_stat_progress_cluster) because I'm not sure whether the

Re: [HACKERS] CLUSTER command progress monitor

2017-09-06 Thread Michael Paquier
On Wed, Sep 6, 2017 at 3:58 PM, Tatsuro Yamada wrote: > I revised the patch like this: You should avoid top-posting. > I didn't change the name of view (pg_stat_progress_cluster) because I'm not > sure > whether the new name (pg_stat_progress_reorg) is suitable or

Re: [HACKERS] CLUSTER command progress monitor

2017-09-06 Thread Tatsuro Yamada
Hi Hackers, I revised the patch like this: - Add "command" column in the view It tells that the running command is CLUSTER or VACUUM FULL. - Enable VACUUM FULL progress monitor Add heap_tuples_vacuumed and heap_tuples_recently_dead as a counter in the view. Sequence of phases

Re: [HACKERS] CLUSTER command progress monitor

2017-09-04 Thread Tatsuro Yamada
On 2017/09/04 15:38, Michael Paquier wrote: On Thu, Aug 31, 2017 at 11:12 AM, Tatsuro Yamada wrote: Then I have questions. * Should we have separate views for them? Or should both be covered by the same view with some indication of which command (CLUSTER

Re: [HACKERS] CLUSTER command progress monitor

2017-09-04 Thread Michael Paquier
On Thu, Aug 31, 2017 at 11:12 AM, Tatsuro Yamada wrote: > Then I have questions. > > * Should we have separate views for them? Or should both be covered by the > same view with some indication of which command (CLUSTER or VACUUM FULL) > is actually

Re: [HACKERS] CLUSTER command progress monitor

2017-09-03 Thread Tatsuro Yamada
Hi Sawada-san, Thanks for taking your time. I'll be more careful. Regards, Tatsuro Yamada On 2017/09/04 11:51, Masahiko Sawada wrote: On Mon, Sep 4, 2017 at 11:37 AM, Tatsuro Yamada wrote: Hi Sawada-san, Thomas, Thanks for sharing the reggression.diff. I

Re: [HACKERS] CLUSTER command progress monitor

2017-09-03 Thread Masahiko Sawada
On Mon, Sep 4, 2017 at 11:37 AM, Tatsuro Yamada wrote: > Hi Sawada-san, Thomas, > > Thanks for sharing the reggression.diff. > I realized Thomas's comment is right. > > Attached patch is fixed version. > Could you try it? > Yeah, in my environment the regression

Re: [HACKERS] CLUSTER command progress monitor

2017-09-03 Thread Tatsuro Yamada
Hi Sawada-san, Thomas, Thanks for sharing the reggression.diff. I realized Thomas's comment is right. Attached patch is fixed version. Could you try it? Regards, Tatsuro Yamada NTT Open Source Software Center On 2017/09/01 17:59, Masahiko Sawada wrote: On Fri, Sep 1, 2017 at 3:38 PM,

Re: [HACKERS] CLUSTER command progress monitor

2017-09-01 Thread Masahiko Sawada
On Fri, Sep 1, 2017 at 3:38 PM, Tatsuro Yamada wrote: > Hi Thomas, > >>> Any comments or suggestion are welcome. >> >> >> Although this patch updates src/test/regress/expected/rules.out I >> think perhaps you included the wrong version? That regression test >> fails

Re: [HACKERS] CLUSTER command progress monitor

2017-09-01 Thread Tatsuro Yamada
Hi Thomas, Any comments or suggestion are welcome. Although this patch updates src/test/regress/expected/rules.out I think perhaps you included the wrong version? That regression test fails for me Thanks for the comment. I use the patch on 7b69b6ce and it's fine. Did you use "initdb"

Re: [HACKERS] CLUSTER command progress monitor

2017-08-31 Thread Thomas Munro
On Thu, Aug 31, 2017 at 2:12 PM, Tatsuro Yamada wrote: > Any comments or suggestion are welcome. Although this patch updates src/test/regress/expected/rules.out I think perhaps you included the wrong version? That regression test fails for me -- Thomas Munro

[HACKERS] CLUSTER command progress monitor

2017-08-30 Thread Tatsuro Yamada
Hi, Following is a proposal for reporting the progress of CLUSTER command: It seems that the following could be the phases of CLUSTER processing: 1. scanning heap 2. sort tuples 3. writing new heap 4. scan heap and write new heap 5. swapping relation files 6. rebuild index 7.