Re: [HACKERS] Default Partition for Range

2017-07-04 Thread Rahila Syed
--Testing the boundary conditions like in the above example following should pass. postgres=# insert into partr_def1 values (1,10); INSERT 0 1 Thank you, Rahila Syed On Mon, Jul 3, 2017 at 8:00 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Fri, Jun 30, 2017 at 11:56 AM,

Re: [HACKERS] Default Partition for Range

2017-06-28 Thread Rahila Syed
Hi Beena, I started testing and reviewing the patch. Can you update the patch as v5 patch does not apply cleanly on master? Thank you, Rahila Syed On Wed, Jun 21, 2017 at 8:43 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Wed, Jun 21, 2017 at 8:08 PM, Robert Haas <robertmh.

Re: [HACKERS] Dropping partitioned table drops a previously detached partition

2017-06-13 Thread Rahila Syed
I reviewed and tested 0001-Dependency-between-partitioned-table-and-partition_v1.patch It applies cleanly on master and make check passes. Following are few comments: >/* > * Drop the dependency created by StoreCatalogInheritance1 (CREATE TABLE > * INHERITS/ALTER TABLE INHERIT -- refclassid will

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-12 Thread Rahila Syed
as a TODO. And run the code >through pgindent I will also update the next version with documentation and regression tests and run pgindent Thank you, Rahila Syed On Fri, May 12, 2017 at 4:33 PM, Beena Emerson <memissemer...@gmail.com> wrote: > > > On Thu, May 11, 2017 at 7:37 PM, R

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-11 Thread Rahila Syed
needed to return false if the node is not DefElem. Todo: Add regression tests Documentation Thank you, Rahila Syed On Fri, May 5, 2017 at 1:30 AM, Jeevan Ladhe <jeevan.la...@enterprisedb.com> wrote: > Hi Rahila, > > I have started reviewing your latest patch, and here are my initia

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-10 Thread Rahila Syed
OK. I get it now. If attach partition already supports scanning all the partitions before attach, similar support should be provided in the case of adding a partition after default partition as well. Thank you, Rahila Syed On Wed, May 10, 2017 at 6:42 AM, Amit Langote <langote_amit...@lab.ntt.co.j

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-09 Thread Rahila Syed
>Hi Rahila, >I am not able add a new partition if default partition is further partitioned >with default partition. >Consider example below: >postgres=# CREATE TABLE test ( a int, b int, c int) PARTITION BY LIST (a); >CREATE TABLE >postgres=# CREATE TABLE test_p1 PARTITION OF test FOR VALUES

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-09 Thread Rahila Syed
+1 for AS DEFAULT syntax if it helps in improving readability specially in following case CREATE TABLE p1 PARTITION OF test AS DEFAULT PARTITION BY LIST(a); Thank you, Rahila Syed On Tue, May 9, 2017 at 1:13 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, May 4, 2017 at 4:40

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-08 Thread Rahila Syed
>I am not able add a new partition if default partition is further partitioned >with default partition. Thanks for reporting. I will fix this. >pg_restore is failing for default partition, dump file still storing old syntax of default partition. Thanks for reporting . I will fix this once the

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-04 Thread Rahila Syed
Hello Amul, Thanks for reporting. Please find attached an updated patch which fixes the above. Also, the attached patch includes changes in syntax proposed upthread. The syntax implemented in this patch is as follows, CREATE TABLE p11 PARTITION OF p1 DEFAULT; Thank you, Rahila Syed On Thu

Re: [HACKERS] Adding support for Default partition in partitioning

2017-05-02 Thread Rahila Syed
Please find attached updated patch with review comments by Robert and Jeevan implemented. The newly proposed syntax CREATE TABLE .. PARTITION OF .. DEFAULT has got most votes on this thread. If there is no more objection, I will go ahead and include that in the patch. Thank you, Rahila Syed

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-27 Thread Rahila Syed
r the default list partitioning patch. After that I can start with default partition for range partitioning. Thank you, Rahila Syed

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-27 Thread Rahila Syed
hand if sub partitions of a default partition is to be prohibited, an error should be thrown if PARTITION BY is specified after DEFAULT. Thank you, Rahila Syed On Tue, Apr 25, 2017 at 1:46 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Apr 24, 2017 at 8:14 AM, Ashutosh Ba

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-26 Thread Rahila Syed
, instead we can simply use an array of oids in rd_partdesc. I think using find_inheritance_children() will take into consideration concurrent drop of a partition which the value in rd_partdesc will not. Thank you, Rahila Syed

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-24 Thread Rahila Syed
ut if (!def_elem) { foreach(cell3, ...) { >... } } instead of reiterating the ReleaseSysCache in two places? Fixed in the attached. I will respond to further comments in following email. On Thu, Apr 13, 2017 at 12:48 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Apr 6, 2017 at 7

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-06 Thread Rahila Syed
Hello, Thanks a lot for testing and reporting this. Please find attached an updated patch with the fix. The patch also contains a fix regarding operator used at the time of creating expression as default partition constraint. This was notified offlist by Amit Langote. Thank you, Rahila Syed

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-05 Thread Rahila Syed
added partition? Thank you, Rahila Syed On Wed, Apr 5, 2017 at 1:22 PM, Amit Langote <langote_amit...@lab.ntt.co.jp> wrote: > On 2017/04/05 14:41, Rushabh Lathia wrote: > > I agree about the future plan about the row movement, how that is I am > > not quite sure at this stage

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-05 Thread Rahila Syed
Hi, >However, running your original example, I'm getting this error Thank you for testing. Please find attached an updated patch which fixes the above. Thank you, Rahila Syed default_partition_v5.patch Description: application/download -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] Adding support for Default partition in partitioning

2017-04-04 Thread Rahila Syed
Hello, Please find attached an updated patch. Following has been accomplished in this update: 1. A new partition can be added after default partition if there are no conflicting rows in default partition. 2. Solved the crash reported earlier. Thank you, Rahila Syed On Tue, Apr 4, 2017 at 6

Re: [HACKERS] Adding support for Default partition in partitioning

2017-03-29 Thread Rahila Syed
Thanks for reporting. I have identified the problem and have a fix. Currently working on allowing adding a partition after default partition if the default partition does not have any conflicting rows. Will update the patch with both of these. Thank you, Rahila Syed On Mon, Mar 27, 2017 at 12:10

Re: [HACKERS] Adding support for Default partition in partitioning

2017-03-24 Thread Rahila Syed
r_list(). Have moved the check inside get_qual_for_partbound() as needed to do some operations before calling get_qual_for_list() for default partitions. Thank you, Rahila Syed On Tue, Mar 21, 2017 at 11:36 AM, Rushabh Lathia <rushabh.lat...@gmail.com> wrote: > I picked this for rev

Re: [HACKERS] Adding support for Default partition in partitioning

2017-03-19 Thread Rahila Syed
patch by tomorrow. Thank you, Rahila Syed On Mon, Mar 13, 2017 at 5:42 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Fri, Mar 10, 2017 at 2:17 PM, Peter Eisentraut > <peter.eisentr...@2ndquadrant.com> wrote: > > On 3/2/17 21:40, Robert Haas wrote: > >> On the po

Re: [HACKERS] wait events for disk I/O

2017-03-16 Thread Rahila Syed
his decision to the committer. Thank you, Rahila Syed On Wed, Mar 15, 2017 at 1:03 PM, Rushabh Lathia <rushabh.lat...@gmail.com> wrote: > Thanks Rahila for reviewing this patch. > > On Tue, Mar 14, 2017 at 8:13 PM, Rahila Syed <rahilasye...@gmail.com> > wrote: > >>

Re: [HACKERS] wait events for disk I/O

2017-03-14 Thread Rahila Syed
in relation to creating dsm segments. write_auto_conf_file()- This is called when updated configuration parameters are written to a temp file. Thank you, Rahila Syed On Wed, Mar 8, 2017 at 4:50 PM, Rushabh Lathia <rushabh.lat...@gmail.com> wrote: &

Re: [HACKERS] Adding support for Default partition in partitioning

2017-03-10 Thread Rahila Syed
ve the relevant tuples from default to the new partition or like you suggest keep monitoring the default partition until user moves the rows out of the default. Thank you, Rahila Syed On Tue, Mar 7, 2017 at 10:00 PM, Keith Fiske <ke...@omniti.com> wrote: > On Thu, Mar 2, 2017 at 9:40 P

[HACKERS] Adding support for Default partition in partitioning

2017-02-28 Thread Rahila Syed
proposal. https://www.postgresql.org/message-id/CAJ3gD9do9o2ccQ7j7+tSgiE1REY65XRiMb= yjo3u3qhyp8e...@mail.gmail.com I am adding it to the current commitfest with the status Waiting on Author as I will submit an updated patch with above ToDos. Kindly give your suggestions. Thank you, Rahila Syed

Re: [HACKERS] Parallel Index-only scan

2017-02-15 Thread Rahila Syed
y_scan in the above condition. IIUC, index->amcanparallel is necessary for parallel index scan to be chosen. We cannot chose parallel index only scan if index_only_scan is happening without worrying about index->amcanparallel value. So OR-ing index->amcanparallel with index_only_scan is probably not

Re: [HACKERS] Parallel Index Scans

2017-01-31 Thread Rahila Syed
advances latest scan which is marked by btscan->btps_arrayKeyCount only when latest scan has ended by checking if(btps_pageStatus == BTPARALLEL_DONE) in _bt_advance_array_keys(). Otherwise, the worker just advances its local so->arrayKeyCount. I hope this provides some clarification. Thank you, Rahila

Re: [HACKERS] Parallel Index Scans

2017-01-31 Thread Rahila Syed
st index pages will give a better estimate of work to be done in parallel. As the amount of work/number of pages divided amongst workers is irrespective of the number of heap pages scanned. Thank you, Rahila Syed On Mon, Jan 30, 2017 at 2:52 PM, Amit Kapila <amit.kapil...@gmail.com> wrote

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-29 Thread Rahila Syed
;cannot set AUTOCOMMIT to %s when inside a transaction\n", newval); >+ } The above change in autocommit behaviour needs to be documented. Thank you, Rahila Syed On Wed, Jan 25, 2017 at 5:48 PM, Rahila Syed <rahilasye...@gmail.com> wrote: > Hello, > > The patch works

Re: [HACKERS] Improvements in psql hooks for variables

2017-01-25 Thread Rahila Syed
Hello, The patch works fine on applying on latest master branch and testing it for various variables as listed in PsqlSettings struct. I will post further comments on patch soon. Thank you, Rahila Syed On Wed, Jan 25, 2017 at 1:33 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > "Danie

Re: [HACKERS] Parallel Index Scans

2017-01-17 Thread Rahila Syed
g in _bt_readnextpage(), if (blkno == P_NONE || !so->currPos.moreRight) { _bt_parallel_done(scan); BTScanPosInvalidate(so->currPos); return false; } So to keep it consistent with the existing code, the check is

Re: [HACKERS] Parallel Index-only scan

2017-01-12 Thread Rahila Syed
orkers Planned: 4 ! -> Parallel Index Only Scan using tenk1_unique1 on tenk1 ! (5 rows) IIUC, parallel operation being performed here is fine as parallel restricted function occurs above Gather node and just the expected output needs to be changed. Thank you, Rahila Syed regress

Re: [HACKERS] Assignment of valid collation for SET operations on queries with UNKNOWN types.

2017-01-03 Thread Rahila Syed
fixes the particular case of CREATE VIEW with literal columns by coercing unknown to text only in this particular case. Are you suggesting extending the patch to include coercing from unknown to text for all possible cases where a column of unknown type is being created? Thank you, Rahila Syed

Re: [HACKERS] Parallel Index Scans

2016-12-23 Thread Rahila Syed
e the function can be modified as follows, + /* + * True indicates that the block number returned is either valid including P_NONE + * and scan is continued or block number is invalid and scan has just + * begun. Thank you, Rahila Syed On Thu, Dec 22, 2016 at 9:49 AM, Amit Kapila <amit.kapil...

Re: [HACKERS] Assignment of valid collation for SET operations on queries with UNKNOWN types.

2016-12-14 Thread Rahila Syed
ding this patch to CF 2017-01 Thank you, Rahila Syed unknown_view_column_to_text_v1.patch Description: application/download -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Assignment of valid collation for SET operations on queries with UNKNOWN types.

2016-12-06 Thread Rahila Syed
llowing queries to run successfully which wasn't the case earlier. postgres=# select a from v UNION select a from v1; a - abc def (2 rows) AND postgres=# select * from v order by 1; a - abc (1 row) Kindly give your opinion. Thank you, Rahila Syed. On Thu, Nov 17, 2016 a

Re: [HACKERS] Improvements in psql hooks for variables

2016-12-02 Thread Rahila Syed
== 0) popt->topt.expanded = 2; else if (value) - popt->topt.expanded = ParseVariableBool(value, param); + { + boolvalid; + boolnewval = ParseVariableBool(value, param, ); + if (valid) Should same variable names (success /

[HACKERS] Assignment of valid collation for SET operations on queries with UNKNOWN types.

2016-11-16 Thread Rahila Syed
ment of a valid collation when the left and the right columns/inputs are UNKNOWN, the above can be resolved. Attached WIP patch does that. Kindly let me know your opinion. Thank you, Rahila Syed invalid_collation_error.patch Description: application/download -- Sent via pgsql-hackers mailin

Re: [HACKERS] Improvements in psql hooks for variables

2016-11-13 Thread Rahila Syed
the return value here. I expect some compilers might >complain under a high level of warnings without this cast, although >TBH if you ask me, I wouldn't know which compiler with which flags >exactly. Thank you for explanation. Thank you, Rahila Syed On Fri, Nov 11, 2016 at 2:57 AM,

Re: [HACKERS] Improvements in psql hooks for variables

2016-10-31 Thread Rahila Syed
* ignore return value */ Sorry for my lack of understanding, can you explain why is above change needed? Thank you, Rahila Syed On Tue, Sep 20, 2016 at 11:16 PM, Daniel Verite <dan...@manitou-mail.org> wrote: > Ashutosh Bapat wrote: > > > You may wa

Re: [HACKERS] Parallel Index Scans

2016-10-18 Thread Rahila Syed
based on log of number of pages so it will make even lesser difference. Thank you, Rahila Syed On Tue, Oct 18, 2016 at 8:38 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Thu, Oct 13, 2016 at 8:48 AM, Amit Kapila <amit.kapil...@gmail.com> > wrote: > > As of now,

Re: [HACKERS] Optimization for lazy_scan_heap

2016-09-25 Thread Rahila Syed
gt;} >vacuum_delay_point(); >next_unskippable_block++; >+ n_skipped++; >} >} Also, instead of incrementing n_skipped everytime, it can be set to 'next_unskippable_block' or 'next_unskippable_block -blkno' once at the end of the loop. Thank you, Rah

Re: [HACKERS] Improvements in psql hooks for variables

2016-09-19 Thread Rahila Syed
iableBool(value, param); ^ In file included from settings.h:12:0, from command.c:50: variables.h:38:7: note: declared here bool ParseVariableBool(const char *value, const char *name, bool *valid); ^ Thank you, Rahila Syed On Mon, Sep 19, 2016 at 11:21 AM, Ashuto

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-15 Thread Rahila Syed
\set AUTOCOMMIT ON in such scenario can be confusing. Thank you, Rahila Syed On Wed, Sep 14, 2016 at 8:06 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Rahila Syed <rahilasye...@gmail.com> writes: > >> Looking at the other variables hooks, they already emit errors and can > &g

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-14 Thread Rahila Syed
l be effective after current transaction has ended. Appropriate message will be displayed notifying this to the user and user need not rerun the set AUTOCOMMIT command. Thank you, Rahila Syed On Thu, Sep 8, 2016 at 9:55 PM, Daniel Verite <dan...@manitou-mail.org> wrote: >

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-08 Thread Rahila Syed
command so it is better placed in exec_command(). Thank you, Rahila Syed On Sat, Sep 3, 2016 at 4:39 PM, Daniel Verite <dan...@manitou-mail.org> wrote: > Rushabh Lathia wrote: > > > It might happen that SetVariable() can be called from other places in > > future

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-02 Thread Rahila Syed
you, Rahila Syed On Fri, Sep 2, 2016 at 3:17 PM, Rushabh Lathia <rushabh.lat...@gmail.com> wrote: > > > On Fri, Sep 2, 2016 at 1:12 PM, Rahila Syed <rahilasye...@gmail.com> > wrote: > >> Hello, >> >> Thank you for comments. >> >> >Above tes

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-02 Thread Rahila Syed
rned on inside a transaction. But only when there is an implicit BEGIN as in following case, postgres=# \set AUTOCOMMIT OFF postgres=# create table test(i int); CREATE TABLE postgres=# \set AUTOCOMMIT ON \set: Cannot set AUTOCOMMIT to ON inside a transaction, either COMMIT or ROLLBACK and retry postgres=#

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-09-01 Thread Rahila Syed
F postgres=# create table test(i int); CREATE TABLE postgres=# \set AUTOCOMMIT ON \set: Cannot set AUTOCOMMIT to ON inside a transaction, either COMMIT or ROLLBACK and retry postgres=# Thank you, Rahila Syed On Wed, Aug 17, 2016 at 6:31 PM, Robert Haas <robertmh...@gmail.com> wrote: > O

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-16 Thread Rahila Syed
nk should be OK as it will be notified via ERROR). So summarizing my view of the discussion on this thread, issuing a psql error seems to be the best option. I will post a patch regarding this if there is no objection. Thank you, Rahila Syed

Re: [HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-08 Thread Rahila Syed
friendliness. While I agree that issuing an ERROR and rolling back the transaction ranks higher in safe behaviour, it is not as common (according to instances stated upthread) as immediately committing any open transaction when switching back to autocommit. Thank you, Rahila Syed On Sun, Aug 7

[HACKERS] Surprising behaviour of \set AUTOCOMMIT ON

2016-08-03 Thread Rahila Syed
rough ecpg commits the ongoing transaction when autocommit is set to ON from OFF. Should such behaviour be implemented for \set AUTOCOMMIT ON as well? Thank you, Rahila Syed

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-03-25 Thread Rahila Syed
>Oops. I forgot to credit you in the commit message. Sorry about that. :-( No problem :). Thanks for the commit. Thank you, Rahila Syed

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-03-24 Thread Rahila Syed
try->st_progress_param[i]); Attached patch fixes this. Thank you, Rahila Syed On Wed, Mar 16, 2016 at 11:30 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Mar 16, 2016 at 6:44 AM, Rahila Syed <rahilasye...@gmail.com> > wrote: > >>Sorta. Committed after renaming wha

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-03-16 Thread Rahila Syed
appear before final vacuum cycle happens for any remaining dead tuples which may span few pages if I am not mistaken. IMO, reporting final count of heap_blks_scanned is correct here, but reporting final heap_blks_vacuumed can happen after the final VACUUM cycle for more accuracy. Thank you, Rahila Syed

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-03-14 Thread Rahila Syed
. Though this function basically updates local structures and sends the count to stat collector via messages we can have a function which will instead modify the shared state using the progress API committed recently. Thank you, Rahila Syed

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-01-29 Thread Rahila Syed
it matches any of the dead tuple tids. So additional parameters for each index can be, scanned_index_tuples total_index_tuples (from pg_class.reltuples entry) Thank you, Rahila Syed [1]. http://www.postgresql.org/message-id/56500356.4070...@bluetreble.com

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-01-28 Thread Rahila Syed
. Yes. This was incorrectly added initially by referring to similar pgstat_report interface functions. Thank you, Rahila Syed On Thu, Jan 28, 2016 at 2:27 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Tue, Jan 26, 2016 at 11:37 PM, Vinayak Pokale <vinpok...@gmail.com> >

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2016-01-08 Thread Rahila Syed
est. Is there still some way to add it to the commitfest 2016-01? As this feature has received lot of feedback in previous commitfest , adding it to this commitfest will surely help in progressing it in order to make it ready for PostgreSQL 9.6. Thank you, Rahila Syed On Mon, Dec 28, 2015 at 6:01 AM, A

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-11-30 Thread Rahila Syed
e of activity flag, reporting only changed parameters to shared memory, ACTIVITY_IS_VACUUM flag being set unnecessarily for ANALYZE and FULL commands ,documentation for new view. Also, finer grain reporting from indexes and heap truncate phase is yet to be incorporated into the patch Thank you, R

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-11-19 Thread Rahila Syed
Hello Michael, I am planning to continue contributing to this feature in any way be it by reviewing the patch or making one. Though I haven't been able to reply to the comments or post an updated patch lately. I plan to do that soon. Thank you, Rahila On Thu, Nov 19, 2015 at 1:09 PM, Michael

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-09-11 Thread Rahila Syed
>This doesn't seem to compile Oh. It compiled successfully when applied on HEAD on my machine. Anyways, the OID is changed to 3309 in the attached patch. 3308 / 3309 both are part of OIDs in unused OID list. Thank you, Rahila Syed Vacuum_progress_checker_v2.patch Description: Binary d

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-31 Thread Rahila Syed
Hello, On Jul 16, 2015 1:48 AM, "Rahila Syed" <rahilasye...@gmail.com> wrote: > > Hello, > > Please find attached updated patch >with an interface to calculate command progress in pgstat.c. This interface currently implements VACUUM progress tracking . I have adde

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-17 Thread Rahila Syed
is stored by each backend as it like. If I understand your point correctly, I think you are missing following, The amount of additional information for each command can be different. We may need an array of text and numeric data to represent more additional information. Thank you, Rahila Syed

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-09 Thread Rahila Syed
counter: heap pages vacuumed. This structure can be accessed by statistics collector to display progress via new view. Thank you, Rahila Syed

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-08-01 Thread Rahila Syed
by summing up heap pages scanned, index pages scanned against total heap pages, total index pages * number of index scans. Thank you, Rahila Syed

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-30 Thread Rahila Syed
dead tuple)/m_w_m,1) This estimates index_scans = 1 if n_dead_tuples = 0 assuming lazy scan heap is likely to find some dead_tuples. If n_dead_tuples is non zero the above estimate gives a lower bound on number of index scans possible. Thank you, Rahila Syed

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-15 Thread Rahila Syed
, Rahila Syed. Vacuum_progress_checker_v1.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-01 Thread Rahila Syed
adding other operations as we work out how (for that operation). Thank you for explaining. This design seems good to me except, adding more than one columns(percent_complete, remaining_time) if required to pg_stat_activity can be less user intuitive than having a separate view for VACUUM. -Rahila

Re: [HACKERS] [PROPOSAL] VACUUM Progress Checker.

2015-07-01 Thread Rahila Syed
://www.postgresql.org/message-id/AANLkTi=TcuMA38oGUKX9p5WVPpY+M3L0XUp7=plt+...@mail.gmail.com Thank you, Rahila Syed On Tue, Jun 30, 2015 at 1:22 PM, Pavel Stehule pavel.steh...@gmail.com wrote: Hi 2015-06-30 9:37 GMT+02:00 Rahila Syed rahilasye...@gmail.com: Hello Hackers, Following

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-03-10 Thread Rahila Syed
Hello, I have some minor comments The comments have been implemented in the attached patch. I think that extra parenthesis should be used for the first expression with BKPIMAGE_HAS_HOLE. Parenthesis have been added to improve code readability. Thank you, Rahila Syed On Mon, Mar 9, 2015 at 5

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-03-02 Thread Rahila Syed
of block references can contain information about presence of data, image , hole and compression. Chunk ID for main data fragments remains as it is . This approach provides for readability and extensibility. Thank you, Rahila Syed On Mon, Mar 2, 2015 at 3:43 PM, Fujii Masao masao.fu...@gmail.com wrote

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2015-02-23 Thread Rahila Syed
nearly same as before. There is some difference in WAL between HEAD and HEAD+patch+compression OFF. This difference corresponds to 1 byte increase with each block reference of xlog record. Thank you, Rahila Syed On Wed, Feb 18, 2015 at 7:53 PM, Syed, Rahila rahila.s...@nttdata.com wrote

Re: [HACKERS] Compression of full-page-writes

2015-01-09 Thread Rahila Syed
. Thank you, Rahila Syed -- View this message in context: http://postgresql.nabble.com/Compression-of-full-page-writes-tp5769039p5833389.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make

Re: [HACKERS] Compression of full-page-writes

2015-01-08 Thread Rahila Syed
® Processor E5-2650 (2 GHz, 8C/16T, 20 MB) * 2 nos RAM: 32GB Disk : HDD 450GB 10K Hot Plug 2.5-inch SAS HDD * 8 nos 1 x 450 GB SAS HDD, 2.5-inch, 6Gb/s, 10,000 rpm Thank you, Rahila Syed -- View this message in context: http://postgresql.nabble.com/Compression-of-full-page-writes

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-18 Thread Rahila Syed
+tgmoaznbuwnls4bpwyqgqteeznoavy7kwdbm0a2-tbarn...@mail.gmail.com Thank you, Rahila Syed On Thu, Dec 18, 2014 at 1:57 PM, Fujii Masao masao.fu...@gmail.com wrote: On Thu, Dec 18, 2014 at 2:21 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec 17, 2014 at 11:33 PM, Rahila Syed rahilasye...@gmail.com

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-17 Thread Rahila Syed
+* as incompressible, complete the block header information as if +* nothing happened. As hole is no longer being compressed, this needs to be changed. Thank you, Rahila Syed On Tue, Dec 16, 2014 at 10:04 PM, Michael Paquier michael.paqu...@gmail.com wrote: On Wed, Dec

[HACKERS] Possibly a comment typo in xlogrecord.h

2014-12-16 Thread Rahila Syed
itself. Thank you, Rahila Syed correct_comment_typo_XLogRecordBlockHeader.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-12 Thread Rahila Syed
records to disk in servers with heavy write operations. So yes improving the performance by saving write volume is a part of the requirement. Thank you, Rahila Syed On Fri, Dec 12, 2014 at 7:48 PM, Bruce Momjian br...@momjian.us wrote: On Fri, Dec 12, 2014 at 08:27:59AM -0500, Robert Haas wrote

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-10 Thread Rahila Syed
specifications: Processors:Intel® Xeon ® Processor E5-2650 (2 GHz, 8C/16T, 20 MB) * 2 nos RAM: 32GB Disk : HDD 450GB 10K Hot Plug 2.5-inch SAS HDD * 8 nos 1 x 450 GB SAS HDD, 2.5-inch, 6Gb/s, 10,000 rpm Thank you, Rahila Syed On Fri, Dec 5, 2014 at 10:38 PM, Robert Haas robertmh

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-10 Thread Rahila Syed
CPU time to compress. However , the total time taken to run 25 transactions for each of the scenario is as follows, compression = 'on' : 1838 secs = 'off' : 1701 secs Different is around 140 secs. Thank you, Rahila Syed On Wed, Dec 10, 2014 at 7:55 PM, Bruce Momjian br

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-07 Thread Rahila Syed
as uncompressed blocks. As you already mentioned, this will need compressing images with hole but we can MemSet hole to 0 in order to make compression of hole less expensive and effective. Thank you, Rahila Syed On Sat, Dec 6, 2014 at 7:37 PM, Michael Paquier michael.paqu...@gmail.com wrote

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-05 Thread Rahila Syed
].compress_len; IIUC, fpi_len in case of compressed block image should be fpi_len = record-blocks[block_id].compress_len; Thank you, Rahila Syed -- View this message in context: http://postgresql.nabble.com/Compression-of-full-page-writes-tp5769039p5829403.html Sent from the PostgreSQL - hackers

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-12-04 Thread Rahila Syed
tests were performed on machine with configuration as follows Server specifications: Processors:Intel® Xeon ® Processor E5-2650 (2 GHz, 8C/16T, 20 MB) * 2 nos RAM: 32GB Disk : HDD 450GB 10K Hot Plug 2.5-inch SAS HDD * 8 nos 1 x 450 GB SAS HDD, 2.5-inch, 6Gb/s, 10,000 rpm Thank you, Rahila Syed

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-11-27 Thread Rahila Syed
as changes.patch. Thank you, Rahila Syed changes.patch Description: Binary data 0002-Support-compression-for-full-page-writes-in-WAL.patch Description: Binary data -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-11-11 Thread Rahila Syed
the records when it is only an attribute of records with backup blocks or the attribute of backup blocks. The current approach is adopted with this thought. Regards, Rahila Syed -- View this message in context: http://postgresql.nabble.com/Compression-of-full-page-writes-tp5769039p5826487.html

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-11-03 Thread Rahila Syed
. This is done in order to avoid adding compression information in WAL record header. Memory allocation on SIGHUP in autovacuum is remaining. Working on it. Thank you, Rahila Syed On Tue, Oct 28, 2014 at 8:51 PM, Rahila Syed rahilasyed...@gmail.com wrote: Do we release the buffers

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-10-28 Thread Rahila Syed
happen in that case? IIUC, instead of reporting PANIC , palloc can be used to allocate memory for uncompressedPages at the time of recovery which will throw ERROR and abort startup process in case of failure. Thank you, Rahila Syed -- View this message in context: http://postgresql.1045698.n5

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-10-16 Thread Rahila Syed
it only during startup as fullPageWrites can be changed at runtime In the attached patch, this check is also added in PostgresMain on SIGHUP after processing postgresql.conf file. Thank you, Rahila Syed On Mon, Sep 29, 2014 at 6:06 PM, Andres Freund and...@anarazel.de wrote: Hi

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-10-09 Thread Rahila Syed
sufficient memory is not available for compression. Thank you, Rahila Syed -- View this message in context: http://postgresql.1045698.n5.nabble.com/Compression-of-full-page-writes-tp5769039p5822391.html Sent from the PostgreSQL - hackers mailing list archive at Nabble.com. -- Sent via

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-22 Thread Rahila Syed
) 6518.681072.34 59.66 Off 501.04 1135.1756.34 The results show around 30 percent decrease in WAL volume due to compression of FPW. Thank you , Rahila Syed Tom

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-19 Thread Rahila Syed
Hello, Maybe. Let's get the basic patch done first; then we can argue about that Please find attached patch to compress FPW using pglz compression. All backup blocks in WAL record are compressed at once before inserting it into WAL buffers . Full_page_writes GUC has been modified to accept

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-19 Thread Rahila Syed
Please find attached patch to compress FPW using pglz compression. Please refer the updated patch attached. The earlier patch added few duplicate lines of code in guc.c file. compress_fpw_v1.patch http://postgresql.1045698.n5.nabble.com/file/n5819659/compress_fpw_v1.patch Thank you, Rahila

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-10 Thread Rahila Syed
I will repeat the above tests with high load on CPU and using the benchmark given by Fujii-san and post the results. Average % of CPU usage at user level for each of the compression algorithm are as follows. CompressionMultipleSingle Off81.1338

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-09-02 Thread Rahila Syed
/08/2014 09:16, Fujii Masao masao.fu...@gmail.com escreveu: On Tue, Aug 19, 2014 at 6:37 PM, Rahila Syed rahilasye...@gmail.com wrote: Hello, Thank you for comments. Could you tell me where the patch for single block in one run is? Please find attached patch for single block

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-08-19 Thread Rahila Syed
Freund and...@2ndquadrant.com wrote: On 2014-08-18 13:06:15 -0400, Robert Haas wrote: On Mon, Aug 18, 2014 at 7:19 AM, Rahila Syed rahilasye...@gmail.com wrote: According to the measurement result, the amount of WAL generated in Multiple Blocks in one run than that in Single Block in one

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-08-19 Thread Rahila Syed
* BkpBlock header + 4 * BLCKSZ. Thank you, On Mon, Aug 18, 2014 at 10:40 PM, Robert Haas robertmh...@gmail.com wrote: On Thu, Jul 3, 2014 at 3:58 PM, Rahila Syed rahilasye...@gmail.com wrote: Updated version of patches are attached. Changes are as follows 1. Improved readability

Fwd: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-08-18 Thread Rahila Syed
1.14 1.16 This shows compression ratios of both the scenarios Multiple blocks and single block are nearly same for this benchmark. Thank you, Rahila Syed

Re: [HACKERS] [REVIEW] Re: Compression of full-page-writes

2014-07-11 Thread Rahila Syed
reaason to deviate from its name. Yes. This will be renamed to full_page_compression according to suggestions earlier in the discussion. Thank you, Rahila Syed On Fri, Jul 11, 2014 at 12:00 PM, Andres Freund and...@2ndquadrant.com wrote: On 2014-07-04 19:27:10 +0530, Rahila Syed wrote

  1   2   >