Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-23 Thread Thakur, Sameer
Hello,
Yes. Any percent completion calculation will have to account for the case of 
needing multiple passes through all the indexes.

Each dead tuple requires 6 bytes (IIRC) of maintenance work mem. So if you're 
deleting 5M rows with m_w_m=1MB you should be getting many passes through the 
indexes. Studying the output of VACUUM VERBOSE will confirm that (or just 
throw a temporary WARNING in the path where we start the scan).

Yes I see the problem now. I get the message WARNING:  Overall index 
percentage completion 100.00 logged  25 times while vacuuming after 5 
million records deleted.
Figuring out number  of multiple index passes beforehand, accurately, is the 
problem to solve. Clearly need to study this some more.
Thank you,
Sameer Thakur | Senior Software Specialist | NTTDATA Global Delivery Services 
Private Ltd | w. +91.20.6641.7146 | VoIP: 8834.8146 | m. +91 989.016.6656 | 
sameer.tha...@nttdata.com | Follow us on Twitter@NTTDATAAmericas


__
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-23 Thread Thakur, Sameer
Hello,
logged  25 times
Sorry, it is much lower at 7 times. Does not change overall point though
regards
Sameer Thakur | Senior Software Specialist | NTTDATA Global Delivery Services 
Private Ltd | w. +91.20.6641.7146 | VoIP: 8834.8146 | m. +91 989.016.6656 | 
sameer.tha...@nttdata.com | Follow us on Twitter@NTTDATAAmericas


__
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-22 Thread Thakur, Sameer
Hello,
I think it'd be better to combine both numbers into one report:
It'd also be good to standardize on where the * 100 is happening.
Done
can be replaced by
(itemptr-ipblkid != vacrelstats-last_scanned_page)
Get compiler error : invalid operands to binary != (have ‘BlockIdData’ and 
‘BlockIdData’)
vacrelstats-current_index_scanned_page_count++;
Done
Please find v3 attached.

I am struggling to create  maintenance work memory exhaustion.  Did the 
following
maintenance_work_mem=1MB.
Inserted 10 million records in tbl1 with 3 indexes. Deleted 5 million and 
vacuumed. So far no error. I could keep bumping up the records to say 100 
million and try to get this error.
This seems a tedious manner to simulate maintenance work memory exhaustion. Is 
there a better way?
To insert I am using COPY (from a csv which has 10 million records) and 
building indexes after insert is complete.
Thank you
Sameer

__
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.


IndexScanProgress_v3.patch
Description: IndexScanProgress_v3.patch

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-20 Thread Thakur, Sameer
Hello,
Does this actually handle multiple indexes? It doesn't appear so, which I'd 
think is a significant problem... :/
Please find v2 attached which does this.
I'm also not seeing how this will deal with exhausting maintenance_work_mem. 
ISTM that when that happens you'd definitely want a better idea of what's 
going on...
Will work on this aspect in v3.
Thank you,
Sameer

__
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.


IndexScanProgress_v2.patch
Description: IndexScanProgress_v2.patch

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers


Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread Thakur, Sameer
Hello,
I am not really willing to show up as the picky guy here, but could it be 
possible to receive those patches as attached to emails instead of having them 
referenced by URL? I imagine that you are directly using the nabble interface.
Just configured a new mail client for nabble, did not know how to use it within 
an existing conversation.
Now I can send the patch attached!
Thanks
Sameer


__
Disclaimer: This email and any attachments are sent in strictest confidence
for the sole use of the addressee and may contain legally privileged,
confidential, and proprietary data. If you are not the intended recipient,
please advise the sender by replying promptly to this email and then delete
and destroy this email and any attachments without any further use, copying
or forwarding.


IndexScanProgress.patch
Description: IndexScanProgress.patch

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers