Re: [HACKERS] ANALYZE command progress checker

> On 05 Apr 2017, at 03:17, Masahiko Sawada wrote: > > On Wed, Apr 5, 2017 at 1:49 AM, Robert Haas wrote: >> On Tue, Apr 4, 2017 at 4:57 AM, Amit Langote >> wrote: >>> Hmm, you're right. It could be counted with a

Re: [HACKERS] ANALYZE command progress checker

On 2017/04/05 10:17, Masahiko Sawada wrote: On Wed, Apr 5, 2017 at 1:49 AM, Robert Haas wrote: On Tue, Apr 4, 2017 at 4:57 AM, Amit Langote wrote: Hmm, you're right. It could be counted with a separate variable initialized to 0 and

Re: [HACKERS] ANALYZE command progress checker

On Wed, Apr 5, 2017 at 1:49 AM, Robert Haas wrote: > On Tue, Apr 4, 2017 at 4:57 AM, Amit Langote > wrote: >> Hmm, you're right. It could be counted with a separate variable >> initialized to 0 and incremented every time we decide to add a

Re: [HACKERS] ANALYZE command progress checker

On Tue, Apr 4, 2017 at 4:57 AM, Amit Langote wrote: > Hmm, you're right. It could be counted with a separate variable > initialized to 0 and incremented every time we decide to add a row to the > final set of sampled rows, although different implementations of >

Re: [HACKERS] ANALYZE command progress checker

On 2017/04/04 15:30, Masahiko Sawada wrote: >> We can report progress in terms of individual blocks only inside >> acquire_sample_rows(), which seems undesirable when one thinks that we >> will be resetting the target for every child table. We should have a >> global target that considers all

Re: [HACKERS] ANALYZE command progress checker

On Tue, Apr 4, 2017 at 2:12 PM, Amit Langote wrote: > On 2017/03/30 17:39, Masahiko Sawada wrote: >> On Wed, Mar 29, 2017 at 5:38 PM, vinayak wrote: > I have updated the patch. >> >> I reviewed v7 patch. >> >> When I ran ANALYZE command to the table having 5

Re: [HACKERS] ANALYZE command progress checker

On 2017/04/04 14:15, Amit Langote wrote: > On 2017/04/04 14:12, Amit Langote wrote: >> Two kinds of statistics are collected if the table is a inheritance parent. >> >> First kind considers the table by itself and calculates regular >> single-table statistics using rows sampled from the only

Re: [HACKERS] ANALYZE command progress checker

On 2017/04/04 14:12, Amit Langote wrote: > Two kinds of statistics are collected if the table is a inheritance parent. > > First kind considers the table by itself and calculates regular > single-table statistics using rows sampled from the only available heap > (by calling do_analyze_rel() with

Re: [HACKERS] ANALYZE command progress checker

On 2017/03/30 17:39, Masahiko Sawada wrote: > On Wed, Mar 29, 2017 at 5:38 PM, vinayak wrote: I have updated the patch. > > I reviewed v7 patch. > > When I ran ANALYZE command to the table having 5 millions rows with 3 > child tables, the progress report I got shows the strange result. The

Re: [HACKERS] ANALYZE command progress checker

On 2017/03/30 17:39, Masahiko Sawada wrote: On Wed, Mar 29, 2017 at 5:38 PM, vinayak wrote: On 2017/03/25 4:30, Robert Haas wrote: On Fri, Mar 24, 2017 at 3:41 AM, vinayak wrote: I have updated the patch. You can't change

Re: [HACKERS] ANALYZE command progress checker

On Thu, Mar 30, 2017 at 6:19 PM, vinayak wrote: > > On 2017/03/30 17:39, Masahiko Sawada wrote: >> >> On Wed, Mar 29, 2017 at 5:38 PM, vinayak >> wrote: >>> >>> On 2017/03/25 4:30, Robert Haas wrote: On Fri, Mar 24, 2017

Re: [HACKERS] ANALYZE command progress checker

On 2017/03/30 17:39, Masahiko Sawada wrote: On Wed, Mar 29, 2017 at 5:38 PM, vinayak wrote: On 2017/03/25 4:30, Robert Haas wrote: On Fri, Mar 24, 2017 at 3:41 AM, vinayak wrote: I have updated the patch. You can't change

Re: [HACKERS] ANALYZE command progress checker

On Wed, Mar 29, 2017 at 5:38 PM, vinayak wrote: > > On 2017/03/25 4:30, Robert Haas wrote: >> >> On Fri, Mar 24, 2017 at 3:41 AM, vinayak >> wrote: >>> >>> I have updated the patch. >> >> You can't change the definition of

Re: [HACKERS] ANALYZE command progress checker

On 2017/03/25 4:30, Robert Haas wrote: On Fri, Mar 24, 2017 at 3:41 AM, vinayak wrote: I have updated the patch. You can't change the definition of AcquireSampleRowsFunc without updating the documentation in fdwhandler.sgml, but I think I don't immediately

Re: [HACKERS] ANALYZE command progress checker

On Fri, Mar 24, 2017 at 3:41 AM, vinayak wrote: > I have updated the patch. You can't change the definition of AcquireSampleRowsFunc without updating the documentation in fdwhandler.sgml, but I think I don't immediately understand why that's a thing we want to do

Re: [HACKERS] ANALYZE command progress checker

On 2017/03/23 15:04, Haribabu Kommi wrote: On Wed, Mar 22, 2017 at 8:11 PM, vinayak > wrote: On 2017/03/21 21:25, Haribabu Kommi wrote: On Tue, Mar 21, 2017 at 3:41 PM, vinayak

Re: [HACKERS] ANALYZE command progress checker

On Wed, Mar 22, 2017 at 8:11 PM, vinayak wrote: > > On 2017/03/21 21:25, Haribabu Kommi wrote: > > > > On Tue, Mar 21, 2017 at 3:41 PM, vinayak > wrote: > >> Thank you for testing the patch on Windows platform. >> >> > Thanks for

Re: [HACKERS] ANALYZE command progress checker

On 2017/03/21 21:25, Haribabu Kommi wrote: On Tue, Mar 21, 2017 at 3:41 PM, vinayak > wrote: Thank you for testing the patch on Windows platform. Thanks for the updated patch. It works good for a normal

Re: [HACKERS] ANALYZE command progress checker

On Tue, Mar 21, 2017 at 3:41 PM, vinayak wrote: > Thank you for testing the patch on Windows platform. > > Thanks for the updated patch. It works good for a normal relation. But for a relation that contains child tables, the PROGRESS_ANALYZE_NUM_ROWS_SAMPLED

Re: [HACKERS] ANALYZE command progress checker

Hi Ashutosh, On 2017/03/19 17:56, Ashutosh Sharma wrote: Hi, I didn't find any major issues with the patch. It works as expected. However, I have few minor comments that I would like to share, + + Total number of sample rows. The sample it reads is taken randomly. + Its size

Re: [HACKERS] ANALYZE command progress checker

Hi Ashutosh, Thank you for reviewing the patch. On 2017/03/18 21:00, Ashutosh Sharma wrote: Hi Vinayak, Here are couple of review comments that may need your attention. 1. Firstly, I am seeing some trailing whitespace errors when trying to apply your v3 patch using git apply command.

Re: [HACKERS] ANALYZE command progress checker

Hi, I didn't find any major issues with the patch. It works as expected. However, I have few minor comments that I would like to share, + + Total number of sample rows. The sample it reads is taken randomly. + Its size depends on the default_statistics_target parameter value. +

Re: [HACKERS] ANALYZE command progress checker

Hi Vinayak, Here are couple of review comments that may need your attention. 1. Firstly, I am seeing some trailing whitespace errors when trying to apply your v3 patch using git apply command. [ashu@localhost postgresql]$ git apply pg_stat_progress_analyze_v3.patch

Re: [HACKERS] ANALYZE command progress checker

On 2017/03/17 10:38, Robert Haas wrote: On Fri, Mar 10, 2017 at 2:46 AM, vinayak wrote: Thank you for reviewing the patch. The attached patch incorporated Michael and Amit comments also. I reviewed this tonight. Thank you for reviewing the patch. +

Re: [HACKERS] ANALYZE command progress checker

On Fri, Mar 10, 2017 at 2:46 AM, vinayak wrote: > Thank you for reviewing the patch. > > The attached patch incorporated Michael and Amit comments also. I reviewed this tonight. +/* Report compute index stats phase */ +

Re: [HACKERS] ANALYZE command progress checker

On Fri, Mar 10, 2017 at 6:46 PM, vinayak wrote: > > + /* Report total number of heap blocks and collectinf sample row phase*/ > + initprog_val[0] = PROGRESS_ANALYZE_PHASE_COLLECT_HEAP_SAMPLE_ROWS; > + initprog_val[1] = totalblocks; > +

Re: [HACKERS] ANALYZE command progress checker

On Fri, Mar 10, 2017 at 2:57 PM, Jim Nasby wrote: > Oh, I wasn't suggesting a single SRF for everything. Hopefully users will > eventually figure out a good formula to drive a "progress bar" for each type > of monitor, which is what you really want anyway (at least 99% of

Re: [HACKERS] ANALYZE command progress checker

On 3/10/17 1:06 PM, Andres Freund wrote: Hi, On 2017-03-10 02:11:18 -0600, Jim Nasby wrote: Perhaps instead of adding more clutter to \dvS we could just have a SRF for now. I don't see that as clutter, it's useful information, and keeping it discoverable is good, not bad. If we keep adding

Re: [HACKERS] ANALYZE command progress checker

Hi, On 2017-03-10 02:11:18 -0600, Jim Nasby wrote: > Perhaps instead of adding more clutter to \dvS we could just have a SRF for > now. I don't see that as clutter, it's useful information, and keeping it discoverable is good, not bad. > At over 2800 rows currently, you're not going to notice

Re: [HACKERS] ANALYZE command progress checker

On 3/6/17 12:49 AM, Michael Paquier wrote: On Sat, Mar 4, 2017 at 5:33 AM, David Steele wrote: I think the idea of a general progress view is very valuable and there are a ton of operations it could be used for: full table scans, index rebuilds, vacuum, copy, etc.

Re: [HACKERS] ANALYZE command progress checker

Thank you for reviewing the patch. The attached patch incorporated Michael and Amit comments also. On 2017/03/07 15:45, Haribabu Kommi wrote: On Tue, Mar 7, 2017 at 5:01 PM, Michael Paquier > wrote: @@ -496,7 +499,6 @@

Re: [HACKERS] ANALYZE command progress checker

On Wed, Mar 1, 2017 at 1:25 PM, Andres Freund wrote: > On 2017-03-01 10:20:41 -0800, David Fetter wrote: >> On Wed, Mar 01, 2017 at 09:45:40AM -0500, Peter Eisentraut wrote: >> > On 2/28/17 04:24, vinayak wrote: >> > > The view provides the information of analyze command

Re: [HACKERS] ANALYZE command progress checker

On Tue, Mar 7, 2017 at 5:01 PM, Michael Paquier wrote: > > @@ -496,7 +499,6 @@ do_analyze_rel(Relation onerel, int options, > VacuumParams *params, > numrows = (*acquirefunc) (onerel, elevel, > rows, targrows, >

Re: [HACKERS] ANALYZE command progress checker

On Mon, Mar 6, 2017 at 3:58 PM, Andres Freund wrote: > On 2017-03-03 15:33:15 -0500, David Steele wrote: >> On 3/1/17 1:25 PM, Andres Freund wrote: >> > On 2017-03-01 10:20:41 -0800, David Fetter wrote: >> >> On Wed, Mar 01, 2017 at 09:45:40AM -0500, Peter Eisentraut wrote: >>

Re: [HACKERS] ANALYZE command progress checker

On 3/6/17 1:58 AM, Andres Freund wrote: > On 2017-03-03 15:33:15 -0500, David Steele wrote: > >> I propose we move this to the 2017-07 CF so the idea can be more fully >> developed. > > I don't see that being warranted in this case, we're really not talking > about something complicated: <...>

Re: [HACKERS] ANALYZE command progress checker

On 2017/03/06 17:02, Amit Langote wrote: Hi Vinayak, On 2017/02/28 18:24, vinayak wrote: The attached patch reports the different phases of analyze command. Added this patch to CF 2017-03. In the updated monitoring.sgml: + + computing heap stats + + VACUUM is currently

Re: [HACKERS] ANALYZE command progress checker

Hi Vinayak, On 2017/02/28 18:24, vinayak wrote: > The attached patch reports the different phases of analyze command. > Added this patch to CF 2017-03. In the updated monitoring.sgml: + + computing heap stats + + VACUUM is currently computing heap stats. + + + +

Re: [HACKERS] ANALYZE command progress checker

On 2017-03-03 15:33:15 -0500, David Steele wrote: > On 3/1/17 1:25 PM, Andres Freund wrote: > > On 2017-03-01 10:20:41 -0800, David Fetter wrote: > >> On Wed, Mar 01, 2017 at 09:45:40AM -0500, Peter Eisentraut wrote: > >>> On 2/28/17 04:24, vinayak wrote: > The view provides the information

Re: [HACKERS] ANALYZE command progress checker

On Sat, Mar 4, 2017 at 5:33 AM, David Steele wrote: > I think the idea of a general progress view is very valuable and there > are a ton of operations it could be used for: full table scans, index > rebuilds, vacuum, copy, etc. > > However, I feel that this proposal is not

Re: [HACKERS] ANALYZE command progress checker

On 3/1/17 1:25 PM, Andres Freund wrote: > On 2017-03-01 10:20:41 -0800, David Fetter wrote: >> On Wed, Mar 01, 2017 at 09:45:40AM -0500, Peter Eisentraut wrote: >>> On 2/28/17 04:24, vinayak wrote: The view provides the information of analyze command progress details as follows

Re: [HACKERS] ANALYZE command progress checker

On March 1, 2017 11:34:48 AM PST, David Fetter wrote: >I notice that that commit has no SGML component. Should it have one? Don't think so. -- Sent from my Android device with K-9 Mail. Please excuse my brevity. -- Sent via pgsql-hackers mailing list

Re: [HACKERS] ANALYZE command progress checker

On Wed, Mar 01, 2017 at 10:28:23AM -0800, Andres Freund wrote: > On 2017-03-01 10:25:49 -0800, Andres Freund wrote: > > We now have a framework for this [1] (currently used by vacuum, but > > extensible). The question is about presentation. I'm fairly sure that > > we shouldn't just add yet

Re: [HACKERS] ANALYZE command progress checker

On 2017-03-01 10:25:49 -0800, Andres Freund wrote: > We now have a framework for this [1] (currently used by vacuum, but > extensible). The question is about presentation. I'm fairly sure that > we shouldn't just add yet another framework, and I doubt that that's > what's proposed by Peter. > >

Re: [HACKERS] ANALYZE command progress checker

On 2017-03-01 10:20:41 -0800, David Fetter wrote: > On Wed, Mar 01, 2017 at 09:45:40AM -0500, Peter Eisentraut wrote: > > On 2/28/17 04:24, vinayak wrote: > > > The view provides the information of analyze command progress details as > > > follows > > > postgres=# \d pg_stat_progress_analyze > >

Re: [HACKERS] ANALYZE command progress checker

On Wed, Mar 01, 2017 at 09:45:40AM -0500, Peter Eisentraut wrote: > On 2/28/17 04:24, vinayak wrote: > > The view provides the information of analyze command progress details as > > follows > > postgres=# \d pg_stat_progress_analyze > >View "pg_catalog.pg_stat_progress_analyze" > >

Re: [HACKERS] ANALYZE command progress checker

On 2/28/17 04:24, vinayak wrote: > The view provides the information of analyze command progress details as > follows > postgres=# \d pg_stat_progress_analyze >View "pg_catalog.pg_stat_progress_analyze" Hmm, do we want a separate "progress" system view for every kind of command?

[HACKERS] ANALYZE command progress checker

Hello Hackers, Following is a proposal for reporting the progress of ANALYZE command: It seems that the following could be the phases of ANALYZE processing: 1. Collecting sample rows 2. Collecting inherited sample rows 3. Computing heap stats 4. Computing index stats 5. Cleaning up indexes The