Re: [HACKERS] Implementing pg_receivewal --no-sync

2017-10-25 Thread Kuntal Ghosh
'to not issue such flush waitings'? + [ 'pg_receivewal', '-D', $stream_dir, '--synchronous', '--no-sync' ], + 'failure if --synchronous specified without --no-sync'); s/without/with -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-ha

Re: [HACKERS] Pluggable storage

2017-10-13 Thread Kuntal Ghosh
tor != NULL); + Assert(slot->tts_storageslotam != NULL); /* passing shouldFree=true for a tuple on a disk page is not sane */ Assert(BufferIsValid(buffer) ? (!shouldFree) : true); For some storage engine, isn't it possible that the buffer is valid and the tuple to be stored is formed in memo

Re: [HACKERS] "inconsistent page found" with checksum and wal_consistency_checking enabled

2017-09-20 Thread Kuntal Ghosh
On Wed, Sep 20, 2017 at 11:05 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Sep 20, 2017 at 2:26 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> On Wed, Sep 20, 2017 at 10:22 AM, Michael Paquier >> <michael.paqu...@gmail.com> wrote:

Re: [HACKERS] "inconsistent page found" with checksum and wal_consistency_checking enabled

2017-09-19 Thread Kuntal Ghosh
and surprised! How come we missed that previously. If page lsns are different, checksums will be different as well. Anyhow, nice catch and thanks for the patch. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hack

Re: [HACKERS] More efficient truncation of pg_stat_activity query strings

2017-09-15 Thread Kuntal Ghosh
report_activity This shows the significance of this patch in terms of performance improvement of pgstat_report_activity. Is there any other tests I should do for the same? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

2017-09-13 Thread Kuntal Ghosh
the > patch, though, not the one I just posted with updates for contrib. > Do you want to look at those? > I've performed the regression tests for the same. It passed all the test cases. Also, I've verified the feature implementation using the queries posted by you earlier and some of my own test c

Re: [HACKERS] pg_stat_wal_write statistics view

2017-09-11 Thread Kuntal Ghosh
On Tue, Sep 12, 2017 at 9:06 AM, Haribabu Kommi <kommi.harib...@gmail.com> wrote: > > > On Tue, Sep 12, 2017 at 2:04 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> > wrote: >> >> On Wed, Sep 6, 2017 at 9:16 AM, Haribabu Kommi <kommi.harib...@gmail.com> >

Re: [HACKERS] pg_stat_wal_write statistics view

2017-09-11 Thread Kuntal Ghosh
to collect all the stats in XLogWrite() irrespective of the type of backend and update the shared counters at once at the end of the function. Thoughts? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@post

Re: [HACKERS] parallelize queries containing initplans

2017-08-09 Thread Kuntal Ghosh
it has to scan the complete t2 relation and send all the tuple to upper node, a worst case for parallelism. Probably, this is the reason the optimizer doesn't pick parallel plan for the above case. Just for clarification, do you see any changes in the plan after forcing parallelism(parallel_tupl

Re: [HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

2017-07-09 Thread Kuntal Ghosh
On Fri, Jul 7, 2017 at 2:53 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Kuntal Ghosh <kuntalghosh.2...@gmail.com> writes: Wow. Thank you for the wonderful explanation. >> On Thu, Jul 6, 2017 at 3:45 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: > >&

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-07 Thread Kuntal Ghosh
On Fri, Jul 7, 2017 at 7:49 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Thu, Jul 6, 2017 at 8:51 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >> On Thu, Jul 6, 2017 at 4:18 PM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> >> wrote: >>>

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-06 Thread Kuntal Ghosh
On Wed, Jul 5, 2017 at 9:35 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Jul 4, 2017 at 4:41 PM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> > wrote: >> I've not yet started the patch and it may take some time for me to >> understand and write

Re: [HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

2017-07-06 Thread Kuntal Ghosh
t only for the first bucket. Do you think my reasoning justifies your concern? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

2017-07-04 Thread Kuntal Ghosh
On Tue, Jul 4, 2017 at 10:56 PM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > On Tue, Jul 4, 2017 at 9:20 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: >> Kuntal Ghosh <kuntalghosh.2...@gmail.com> writes: >>> On Tue, Jul 4, 2017 at 9:23 AM, Tom Lane <t..

Re: [HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

2017-07-04 Thread Kuntal Ghosh
On Tue, Jul 4, 2017 at 9:20 PM, Tom Lane <t...@sss.pgh.pa.us> wrote: > Kuntal Ghosh <kuntalghosh.2...@gmail.com> writes: >> On Tue, Jul 4, 2017 at 9:23 AM, Tom Lane <t...@sss.pgh.pa.us> wrote: >>> ... I have to admit that I've failed to wrap my brain

Re: [HACKERS] WIP patch: distinguish selectivity of < from <= and > from >=

2017-07-04 Thread Kuntal Ghosh
tion, The correction is calculated as = 1 / num_distinct_values = .001. Since, the thousand column of tenk1 is uniformly distributed, this turns out to be the exact selectivity. (rows = .001 * 1000 = 10) Thoughts? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-04 Thread Kuntal Ghosh
On Tue, Jul 4, 2017 at 12:44 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Tue, Jul 4, 2017 at 3:25 PM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> > wrote: >> On Mon, Jul 3, 2017 at 6:50 PM, Michael Paquier >> <michael.paqu...@gmail.com> wrote:

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-04 Thread Kuntal Ghosh
ter be int64. Thank you. I'll do the changes and submit the revised patch. I've added an entry in commitfest for the same. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your su

Re: [HACKERS] Error while copying a large file in pg_rewind

2017-07-04 Thread Kuntal Ghosh
is broken for relation files higher than 2GB, > see fetch_file_range where the begin location is an int32. > -- Okay. So, if the relation block size differs by 2GB or more between the source and target directory, we've a problem. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Error while copying a large file in pg_rewind

2017-07-03 Thread Kuntal Ghosh
chchunks, line 2659, column begin: "214800" I guess we've to change the data type to bigint. Also, we need some implementation of ntohl() for 8-byte data types. I've attached a script to reproduce the error and a draft patch. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: h

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-23 Thread Kuntal Ghosh
On Fri, Jun 23, 2017 at 3:01 AM, Thomas Munro <thomas.mu...@enterprisedb.com> wrote: > On Thu, Jun 22, 2017 at 4:29 AM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> On Wed, Jun 21, 2017 at 7:52 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >>>

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Kuntal Ghosh
On Thu, Jun 22, 2017 at 9:48 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Thu, Jun 22, 2017 at 1:29 AM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> On Wed, Jun 21, 2017 at 7:52 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: >>> On Wed

Re: [HACKERS] An attempt to reduce WALWriteLock contention

2017-06-21 Thread Kuntal Ghosh
roach to > group flushes. As of now, I've no plans to re-submit the patch. Actually, I'm not sure what I should try next. I would love to get some advice/direction regarding this. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing li

Re: [HACKERS] An attempt to reduce WALWriteLock contention

2017-06-21 Thread Kuntal Ghosh
k patch alone. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Kuntal Ghosh
On Wed, Jun 21, 2017 at 7:52 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Wed, Jun 21, 2017 at 7:47 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >>> IMHO, It's not a good idea to use DSM call to verify the DSA handle. >>> >> Okay. Is th

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Kuntal Ghosh
On Wed, Jun 21, 2017 at 7:07 PM, Dilip Kumar <dilipbal...@gmail.com> wrote: > On Wed, Jun 21, 2017 at 6:50 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> I think we can just check dsm_find_mapping() to check whether the dsm >> handle is al

Re: [HACKERS] Incorrect documentation about pg_stat_activity

2017-06-21 Thread Kuntal Ghosh
On Wed, Jun 21, 2017 at 6:05 PM, Yugo Nagata <nag...@sraoss.co.jp> wrote: > > Attached is a patch for the documentation fix. > Please attach the patch as well. :-) -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers

Re: [HACKERS] Autovacuum launcher occurs error when cancelled by SIGINT

2017-06-21 Thread Kuntal Ghosh
g like, } - else + else if(!dsm_find_mapping(AutoVacuumShmem->av_dsa_handle)) { AutoVacuumDSA = dsa_attach(AutoVacuumShmem->av_dsa_handle); dsa_pin_mapping(AutoVacuumDSA); Thoughts? -- Thanks & Regards, Kuntal Ghosh Enterpr

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-06-03 Thread Kuntal Ghosh
d background workers, in private memory). Instead, we use only BackgroundWorkerSlots. Perhaps, this is the reason that backend_type is NULL for parallel workers. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hacke

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-06-03 Thread Kuntal Ghosh
where only the worker type is used. The > concatenation just doesn't sit well with me, especially if it requires > the bgw_name_extra to start with space. > +1. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "create publication..all tables" ignore 'partition not supported' error

2017-06-01 Thread Kuntal Ghosh
On Thu, Jun 1, 2017 at 6:56 AM, Peter Eisentraut <peter.eisentr...@2ndquadrant.com> wrote: > On 5/31/17 02:17, Kuntal Ghosh wrote: >> On Wed, May 31, 2017 at 12:58 AM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >>> >>> I'd say we can fix th

Re: [HACKERS] "create publication..all tables" ignore 'partition not supported' error

2017-05-31 Thread Kuntal Ghosh
e a check for the fourth condition as well. Added to open items. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Create subscription with `create_slot=false` and incorrect slot name

2017-05-23 Thread Kuntal Ghosh
_name can be provided even when create_slot is set false, it should be validated as well while creating the subscription. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] pg_dump ignoring information_schema tables which used in Create Publication.

2017-05-22 Thread Kuntal Ghosh
On Mon, May 22, 2017 at 5:22 PM, tushar <tushar.ah...@enterprisedb.com> wrote: > On 05/22/2017 05:12 PM, Kuntal Ghosh wrote: >> >> pg_dump ignores anything created under object name "pg_*" or >> "information_schema". > > In this below scenari

Re: [HACKERS] Increasing parallel workers at runtime

2017-05-22 Thread Kuntal Ghosh
we SHOULD start with n number of workers. However, error in selectivity estimation(The root of all evil, the Achilles Heel of query optimization, according to Guy Lohman et al. :)) can always prove the optimizer wrong. In that case, +1 for your suggested approach of dynamically add or kill some workers

Re: [HACKERS] pg_dump ignoring information_schema tables which used in Create Publication.

2017-05-22 Thread Kuntal Ghosh
.enterprisedb.com/ > The Enterprise PostgreSQL Company > > > > -- > Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) > To make changes to your subscription: > http://www.postgresql.org/mailpref/pgsql-hackers -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] "create publication..all tables" ignore 'partition not supported' error

2017-05-22 Thread Kuntal Ghosh
Partition key: RANGE (logdate) > Publications: > "p" > > Publication 'p' has been set against partition table ,which is not > supported. > > -- > regards,tushar > EnterpriseDB https://www.enterprisedb.com/ > The Enterprise PostgreSQL Company > > >

Re: [HACKERS] Server Crashes if try to provide slot_name='none' at the time of creating subscription.

2017-05-16 Thread Kuntal Ghosh
Added to open item lists. On Tue, May 16, 2017 at 6:35 AM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Mon, May 15, 2017 at 8:09 PM, Masahiko Sawada <sawada.m...@gmail.com> > wrote: >> On Mon, May 15, 2017 at 8:06 PM, Kuntal Ghosh >> <kuntalghosh.2...@gma

Re: [HACKERS] Server Crashes if try to provide slot_name='none' at the time of creating subscription.

2017-05-15 Thread Kuntal Ghosh
On Mon, May 15, 2017 at 5:04 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Mon, May 15, 2017 at 8:22 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> On Mon, May 15, 2017 at 4:39 PM, Masahiko Sawada <sawada.m...@gmail.com> >> wrote: >>

Re: [HACKERS] Server Crashes if try to provide slot_name='none' at the time of creating subscription.

2017-05-15 Thread Kuntal Ghosh
ecoding found initial starting point at 0/162DF18 DETAIL: Waiting for transactions (approximately 1) older than 560 to end. And, it hangs. Is this an expected behavior? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql

Re: [HACKERS] Server Crashes if try to provide slot_name='none' at the time of creating subscription.

2017-05-15 Thread Kuntal Ghosh
blisher: could not connect to server: Connection refused Is the server running locally and accepting connections on Unix domain socket "/tmp/.s.PGSQL.5432"? Changing the order of subscription parameter changes the output. I think the modifications should be done at the

Re: [HACKERS] Create publication syntax is not coming properly in pg_dump / pg_dumpall

2017-05-15 Thread Kuntal Ghosh
you for reporting. > > Hm, It's a bug of pg_dump. Attached patch should fix both pg_dump and > pg_dumpall. > I've reproduced the bug and the patch fixes the issue for me. Also, tested with different combinations of insert, delete and update. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Why does logical replication launcher set application_name?

2017-04-11 Thread Kuntal Ghosh
t; backend_type to guess what is this bgworker. > > Wait, why do we need two ways? > For backend_type=background worker, application_name shows the name of the background worker (BackgroundWorker->bgw_name). I think we need some way to distinguish among different background workers. B

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-11 Thread Kuntal Ghosh
anupBackgroundWorker() - ReportBackgroundWorkerExit() - ForgetBackgroundWorker() But, I'm not sure for any other cases. Should we include the assert statement as well? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-10 Thread Kuntal Ghosh
esult, the difference in unsigned integer will be near UINT32_MAX. Hence, we need the absolute difference after typecasting the same to integer. This value should be less than the max_parallel_workers upper limit. I've attached both the patches for better accessibility. PFA. -- Thanks

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-06 Thread Kuntal Ghosh
On Wed, Apr 5, 2017 at 6:49 PM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Wed, Apr 5, 2017 at 12:35 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> On Tue, Apr 4, 2017 at 11:22 PM, Tomas Vondra >>> I'm probably missing something, but I don't quit

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-05 Thread Kuntal Ghosh
On Wed, Apr 5, 2017 at 7:45 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Apr 5, 2017 at 10:09 AM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >>> Did you intend to attach that patch to this email? >>> >> Actually, I'm con

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-05 Thread Kuntal Ghosh
On Wed, Apr 5, 2017 at 7:31 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Apr 5, 2017 at 6:36 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> > wrote: >> Yes. But, as Robert suggested up in the thread, we should not use >> (parallel_register_count =

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-05 Thread Kuntal Ghosh
fore BackgroundWorkerArray claims this > is the case. > Agreed on the overflowed case. But, my concern is when an overflow has not yet happened: Suppose, uint parallel_register_count = 1; /* Didn't overflow* / uint parallel_terminate_count = 2; /* Didn't overflow */ Assert(parallel_regis

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-05 Thread Kuntal Ghosh
On Wed, Apr 5, 2017 at 3:07 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > > > On 04/05/2017 09:05 AM, Kuntal Ghosh wrote: >> >> AFAICU, during crash recovery, we wait for all non-syslogger children >> to exit, then reset shmem(call Backg

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-05 Thread Kuntal Ghosh
t will bring down the server unnecessarily while executing q2. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-05 Thread Kuntal Ghosh
On Tue, Apr 4, 2017 at 11:22 PM, Tomas Vondra <tomas.von...@2ndquadrant.com> wrote: > On 04/04/2017 06:52 PM, Robert Haas wrote: >> >> On Mon, Apr 3, 2017 at 6:08 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> >> wrote: >>> >>> On Fri, Mar 31,

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-04-03 Thread Kuntal Ghosh
On Fri, Mar 31, 2017 at 6:50 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Mar 30, 2017 at 4:35 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> 2. the server restarts automatically, initialize >> BackgroundWorkerData->parallel_register_

Re: [HACKERS] increasing the default WAL segment size

2017-03-30 Thread Kuntal Ghosh
On Fri, Mar 31, 2017 at 10:40 AM, Beena Emerson <memissemer...@gmail.com> wrote: > On 30 Mar 2017 15:10, "Kuntal Ghosh" <kuntalghosh.2...@gmail.com> wrote: >> 03-modify-tools.patch - Makes XLogSegSize into a variable, currently set >> as >> XLOG_SEG_SI

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-03-30 Thread Kuntal Ghosh
On Fri, Mar 31, 2017 at 5:43 AM, Neha Khatri <nehakhat...@gmail.com> wrote: > > On Fri, Mar 31, 2017 at 8:29 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> > wrote: >> >> On Fri, Mar 31, 2017 at 2:05 AM, Kuntal Ghosh >> <kuntalghosh.2...@gmai

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-03-30 Thread Kuntal Ghosh
On Fri, Mar 31, 2017 at 2:05 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > > 1. Put an Assert(0) in ParallelQueryMain(), start server and execute > any parallel query. > In LaunchParallelWorkers, you can see >nworkers = n nworkers_launched = n (n>0

Re: [HACKERS] strange parallel query behavior after OOM crashes

2017-03-30 Thread Kuntal Ghosh
ckgroundWorkerData->parallel_terminate_count) >= max_parallel_workers) DO NOT launch any parallel worker. Hence, nworkers = n nworkers_launched = 0. I thought because of my stupid mistake the parallel worker is crashing, so, this is supposed to happen. Sorry for that. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] increasing the default WAL segment size

2017-03-30 Thread Kuntal Ghosh
s of tests. > > > 05-initdb_tests.patch adds tap tests to initialize cluster with different > wal_segment_size and then check the config values. What other tests do you > have in mind? Checking the various tools? Nothing from me to add here. I've nothing to add here for the attached

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-27 Thread Kuntal Ghosh
Thank you Robert for committing the patch. commit fc70a4b0df38bda6a13941f1581f25fbb643c7f3 I've changed the status to Committed. On Mon, Mar 27, 2017 at 6:09 AM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Sat, Mar 25, 2017 at 5:26 PM, Kuntal Ghosh > <kuntalghosh

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-25 Thread Kuntal Ghosh
On Fri, Mar 24, 2017 at 9:23 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Mar 23, 2017 at 7:29 AM, Michael Paquier > <michael.paqu...@gmail.com> wrote: >> On Thu, Mar 23, 2017 at 8:19 PM, Kuntal Ghosh >> <kuntalghosh.2...@gmail.com> wrote: >

Re: [HACKERS] BUG: pg_dump generates corrupted gzip file in Windows

2017-03-24 Thread Kuntal Ghosh
On Fri, Mar 24, 2017 at 2:17 PM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > On Fri, Mar 24, 2017 at 12:35 PM, Craig Ringer <cr...@2ndquadrant.com> wrote: >> On 24 March 2017 at 14:07, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: >>> On Fri, Ma

Re: [HACKERS] BUG: pg_dump generates corrupted gzip file in Windows

2017-03-24 Thread Kuntal Ghosh
On Fri, Mar 24, 2017 at 12:35 PM, Craig Ringer <cr...@2ndquadrant.com> wrote: > On 24 March 2017 at 14:07, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: >> On Fri, Mar 24, 2017 at 11:28 AM, Kuntal Ghosh >> <kuntalghosh.2...@gmail.com> wrote: >>> H

Re: [HACKERS] BUG: pg_dump generates corrupted gzip file in Windows

2017-03-24 Thread Kuntal Ghosh
On Fri, Mar 24, 2017 at 11:28 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > Hello, > In Windows, if one needs to take a dump in plain text format (this is > the default option, or can be specified using -Fp) with some level of > compression (-Z[0-9]), an output file has

[HACKERS] BUG: pg_dump generates corrupted gzip file in Windows

2017-03-23 Thread Kuntal Ghosh
, fmt is set to archNull. In that case, the binary mode will not be forced(I think). To fix this, I've attached a patch which adds one extra check in the 'if condition' to check the compression level. PFA. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com 0001-Fix-p

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-23 Thread Kuntal Ghosh
tartup StartupProcessMain() walwriter WalWriterMain() walreceiver WalReceiverMain() walsender InitWalSender() Hence, to be consistent with others, bgworker processes can be initialized from BackgroundWorkerInitializeConnectionBy[Oid]. I've attached the updated patches which reflect the above change. PFA.

Re: [HACKERS] increasing the default WAL segment size

2017-03-22 Thread Kuntal Ghosh
that the problem is in following code: /* parse files as start/end boundaries, extract path if not specified */ if (optind < argc) { + if (!RetrieveXLogSegSize(full_path)) ... } In this case, RetrieveXLogSegSize is conditionally called. So, if the condition is false, XLogSegSize will not be init

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-21 Thread Kuntal Ghosh
visible to all user for now. >> >> Please find the updated patches in the attachment. > > Yeah, hiding it may make sense... Modified. > /* The autovacuum launcher is done here */ > if (IsAutoVacuumLauncherProcess()) > + { > return; > + } > Unne

Re: [HACKERS] Two phase commit in ECPG

2017-03-17 Thread Kuntal Ghosh
On Fri, Mar 17, 2017 at 4:34 PM, Masahiko Sawada <sawada.m...@gmail.com> wrote: > On Fri, Mar 17, 2017 at 12:17 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> On Tue, Mar 14, 2017 at 1:35 AM, Michael Meskes <mes...@postgresql.org> >> wrote: &

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-17 Thread Kuntal Ghosh
On Thu, Mar 16, 2017 at 1:18 PM, Michael Paquier <michael.paqu...@gmail.com> wrote: > On Wed, Mar 15, 2017 at 9:14 PM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> I've attached the updated patches. > > Thanks for the new versions. This begins to l

Re: [HACKERS] Two phase commit in ECPG

2017-03-17 Thread Kuntal Ghosh
nsactions accordingly fixes the issue. But, I'm not sure whether this test should be performed by installcheck by default or should only be performed by make installcheck-prepared-txns. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hack

Re: [HACKERS] parallelize queries containing initplans

2017-03-15 Thread Kuntal Ghosh
ose cases if required in a separate patch. +1. Unfortunately, this patch doesn't enable parallelism for all possible cases with InitPlans. Our objective is to keep things simple and clean. Still, TPC-H q22 runs 2.5~3 times faster with this patch. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB:

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-15 Thread Kuntal Ghosh
pg_stat_get_backend_pid(s.backendid) AS pid, + |pg_stat_get_backend_activity(s.backendid) AS query + | FROM (SELECT pg_stat_get_backend_idset() AS backendid) AS s; 16925 | 16927 | 16926 | 16929 | IMHO, this scenario can be easily avoided by filterin

Re: [HACKERS] WAL Consistency checking for hash indexes

2017-03-14 Thread Kuntal Ghosh
>> +heap2, btree, gin, >> +gist, sequence, spgist, >> +brin, and generic. Only > > Did that, committed this. Also ran pgindent over hash_mask() and > fixed an instance of dubious capitalization. Thanks Robert for the commit. -- Thanks & Regards, Kuntal

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-14 Thread Kuntal Ghosh
is useful for generating a set of currently active client backend ID numbers (from 1 to the number of active client backends). These IDs are used for some pgstat_* functions relevant to client processes, e.g., pg_stat_get_backend_activity, pg_stat_get_backend_client_port etc. Any suggestions? -- Th

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-09 Thread Kuntal Ghosh
| autovacuum launcher Activity| LogicalLauncherMain | idle | bgworker Activity| WalSenderMain | idle | wal sender | | active | client backend Activity| BgWriterMain| idle | writer Activity | Checkpoin

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-03-09 Thread Kuntal Ghosh
) > Fixed. > In 0002: > > @@ -248,6 +248,9 @@ BackgroundWriterMain(void) > */ > prev_hibernate = false; > > +/* report walwriter process in the PgBackendStatus array */ > +pgstat_procstart(); > + > > s/walwriter/writer/g Fixed

Re: [HACKERS] WAL Consistency checking for hash indexes

2017-03-08 Thread Kuntal Ghosh
On Fri, Mar 3, 2017 at 9:44 AM, Amit Kapila <amit.kapil...@gmail.com> wrote: > On Tue, Feb 28, 2017 at 11:06 AM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> Hello everyone, >> >> I've attached a patch which implements WAL consistency checking for &g

Re: [HACKERS] Performance degradation in TPC-H Q18

2017-03-05 Thread Kuntal Ghosh
ncy on a busy system; I'm fairly strongly of the opinion > that you ought to downgrade that by a couple of levels, say to DEBUG3 > or so. +1 I've tested with TPC-H query 18 and it's working fine. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com

Re: [HACKERS] Performance degradation in TPC-H Q18

2017-03-02 Thread Kuntal Ghosh
ther approach, but it would require an extra shift op every time we want to find the next or prev location during a collision. [1] https://www.postgresql.org/message-id/caeepm%3d3rdgjfxw4ckvj0oemya2-34b0qhng1xv0vk7tgpjg...@mail.gmail.com -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://ww

Re: [HACKERS] Performance degradation in TPC-H Q18

2017-02-28 Thread Kuntal Ghosh
On Wed, Mar 1, 2017 at 10:53 AM, Andres Freund <and...@anarazel.de> wrote: > On 2017-03-01 10:47:45 +0530, Kuntal Ghosh wrote: >> if (insertdist > curdist) >> { >> swap the entry to be inserted with the current entry. >> Try to insert the current entry in the s

Re: [HACKERS] Performance degradation in TPC-H Q18

2017-02-28 Thread Kuntal Ghosh
On Wed, Mar 1, 2017 at 9:38 AM, Andres Freund <and...@anarazel.de> wrote: > Hi, > > On 2017-03-01 09:33:07 +0530, Kuntal Ghosh wrote: >> On Wed, Mar 1, 2017 at 9:19 AM, Andres Freund <and...@anarazel.de> wrote: >> >> So, I was looking for other

Re: [HACKERS] Performance degradation in TPC-H Q18

2017-02-28 Thread Kuntal Ghosh
On Wed, Mar 1, 2017 at 9:33 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> wrote: > On Wed, Mar 1, 2017 at 9:19 AM, Andres Freund <and...@anarazel.de> wrote: >> That's without the patch in >> http://archives.postgresql.org/message-id/20161123083351.5vramz52n

Re: [HACKERS] Performance degradation in TPC-H Q18

2017-02-28 Thread Kuntal Ghosh
On Wed, Mar 1, 2017 at 9:19 AM, Andres Freund <and...@anarazel.de> wrote: > On 2017-03-01 09:13:15 +0530, Kuntal Ghosh wrote: >> On Wed, Mar 1, 2017 at 8:48 AM, Andres Freund <and...@anarazel.de> wrote: >> >> BTW, another option to consider is lowering the targ

Re: [HACKERS] Performance degradation in TPC-H Q18

2017-02-28 Thread Kuntal Ghosh
nt probe length for the element being inserted, swap the two elements and keep going. It leads to a low variance for the chain lengths rather than the last one. Is this approach already considered? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via p

[HACKERS] WAL Consistency checking for hash indexes

2017-02-27 Thread Kuntal Ghosh
://www.postgresql.org/message-id/CAA4eK1%2B%2BP%2BjVZC7MC3xzw5uLCpva9%2BgsBpd3semuWffWAftr5Q%40mail.gmail.com -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com 0001-wal_consistency_checking-for-hash-index.patch Description: application/download -- Sent via pgsql-hac

Re: [HACKERS] increasing the default WAL segment size

2017-02-27 Thread Kuntal Ghosh
On Tue, Feb 28, 2017 at 9:45 AM, Jim Nasby <jim.na...@bluetreble.com> wrote: > On 2/24/17 6:30 AM, Kuntal Ghosh wrote: >> >> * You're considering any WAL file with a power of 2 as valid. Suppose, >> the correct WAL seg size is 64mb. For some reason, the server >>

Re: [HACKERS] increasing the default WAL segment size

2017-02-24 Thread Kuntal Ghosh
r documentation describing the scope and limitations of each approach? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

[HACKERS] Performance degradation in TPC-H Q18

2017-02-21 Thread Kuntal Ghosh
en when the filled percentage is pretty low, causing unnecessary memory consumption. Thoughts? [1] https://www.postgresql.org/message-id/20161123083351.5vramz52nmdokhzz%40alap3.anarazel.de 0001-Resize-simplehash.h-table-in-case-of-long-runs.patch -- Thanks & Regards, Kuntal Gh

Re: [HACKERS] Passing query string to workers

2017-02-20 Thread Kuntal Ghosh
y_data = estate->es_sourceText; estate->es_sourceText is a const char* variable. Assigning this const pointer to a non-const pointer violates the rules constant-correctness. So, either you should change query_data as const char*, or as Robert suggested, you can directly use est

Re: [HACKERS] Passing query string to workers

2017-02-16 Thread Kuntal Ghosh
mode). Query is getting displayed for parallel workers in pg_stat_activity, log statements and failed-query statements. Moved status to Ready for committer. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hacker

Re: [HACKERS] Passing query string to workers

2017-02-16 Thread Kuntal Ghosh
ty(STATE_RUNNING, shm_toc_lookup(toc, PARALLEL_KEY_QUERY_TEXT)); Just one lookup is sufficient. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] increasing the default WAL segment size

2017-02-16 Thread Kuntal Ghosh
oth the patches needs a rebase based on the commit 85c11324cabaddcfaf3347df7 (Rename user-facing tools with "xlog" in the name to say "wal"). -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-16 Thread Kuntal Ghosh
NULL, NULL, NULL + }, Maximum value for vacuum_cleanup_index_scale_factor should be 1 instead of 100. As the code indicates, it is certainly not treated as a percentage fraction of relpages. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent

Re: [HACKERS] Write Ahead Logging for Hash Indexes

2017-02-15 Thread Kuntal Ghosh
9e9ceec7cef1. -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] exposing wait events for non-backends (was: Tracking wait event for latches)

2017-02-15 Thread Kuntal Ghosh
; Thoughts? -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com 0001-Infra-to-expose-non-backend-processes-in-pg_stat_get.patch Description: application/download 0002-Expose-stats-for-auxiliary-processes-in-pg_stat_get_.patch Description: application/download 0003-Ex

Re: [HACKERS] GUC for cleanup indexes threshold.

2017-02-10 Thread Kuntal Ghosh
hose index entries later? (I'm a newbie.) +This parameter can only be set anywhere. Oxymoron. :-) -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] Passing query string to workers

2017-02-10 Thread Kuntal Ghosh
0007). -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] WAL consistency check facility

2017-02-08 Thread Kuntal Ghosh
On Thu, Feb 9, 2017 at 2:26 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Wed, Feb 8, 2017 at 1:25 AM, Kuntal Ghosh <kuntalghosh.2...@gmail.com> > wrote: >> Thank you Robert for the review. Please find the updated patch in the >> attachment. > > I ha

Re: [HACKERS] WAL consistency check facility

2017-02-07 Thread Kuntal Ghosh
On Tue, Jan 31, 2017 at 9:36 PM, Robert Haas <robertmh...@gmail.com> wrote: > On Thu, Jan 5, 2017 at 12:54 AM, Kuntal Ghosh > <kuntalghosh.2...@gmail.com> wrote: >> I've attached the patch with the modified changes. PFA. > > Can this patch check contrib/bloom? > A

Re: [HACKERS] WAL consistency check facility

2017-01-31 Thread Kuntal Ghosh
d as well by frontends, this makes the header > dependencies cleaner. (I have looked at using Page when hacking this > stuff, but the header dependencies are not worth it, I don't recall > all the details though this was a couple of months back). + 1 -- Thanks & Regards, Kuntal Ghosh EnterpriseDB: http://www.enterprisedb.com -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers

  1   2   >