Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Alvaro Herrera]

Robert Haas wrote:

> It might be better to document this in bgworker.sgml instead.  That
> already documents some facts about exiting:
> 
>   
>If bgw_restart_time for a background worker is
>configured as BGW_NEVER_RESTART, or if it exits with an exit
>code of 0 or is terminated by TerminateBackgroundWorker,
>it will be automatically unregistered by the postmaster on exit.
>Otherwise, it will be restarted after the time period configured via
>bgw_restart_time, or immediately if the postmaster
>reinitializes the cluster due to a backend failure.  Backends which need
>to suspend execution only temporarily should use an interruptible sleep
>rather than exiting; this can be achieved by calling
>WaitLatch(). Make sure the
>WL_POSTMASTER_DEATH flag is set when calling that function, and
>verify the return code for a prompt exit in the emergency case that
>postgres itself has terminated.
>   
> 
> That paragraph leaves out a number of important details, like:
> 
> 1. A background worker that exits with any exit code other than 0 or 1
> will cause a postmaster crash-and-restart cycle.
> 
> 2. Therefore, an exit of code 1 should be used whenever the process
> wants to be restarted in accordance with bgw_restart_time, and is
> therefore in some sense the "normal" way for a background worker to
> exit.

Yeah, I think bgworker.sgml is the right place to document these
details.

> 3. The aforementioned details about how 9.3 behavior was different
> from current behavior.

Not really sure about this.  I think patching the 9.3 docs to state the
details differently from the 9.4--master details would not be
sufficient, as people moving code would not read the 9.4 docs again to
ensure the semantics remain the same (and since 9.4 has been out for
awhile, patching the release notes wouldn't suffice either).  Patching
the 9.3 docs to say "9.4 is different" would be odd, since 9.4 is "in
the future" for the POV of 9.3 docs.

I think the best option is to backpatch a doc change from 9.4 onwards
stating what is the new behavior, and add a note stating that 9.3 was
different.

-- 
Álvaro Herrerahttp://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Robert Haas]

On Wed, Jun 29, 2016 at 4:35 PM, Piotr Stefaniak
 wrote:
> On 2016-06-29 18:58, Robert Haas wrote:
>> This code predates be7558162acc5578d0b2cf0c8d4c76b6076ce352, prior to
>> which proc_exit(0) forced an immediate, unconditional restart.  It's
>> true that, given that commit, changing this code to do proc_exit(0)
>> instead of proc_exit(1) would be harmless.  However, people writing
>> background workers that might need to work with 9.3 would be best
>> advised to stick with proc_exit(1).  Therefore, I maintain that this
>> is not broken and doesn't need to be fixed.
>
> Then I'd argue that it ought to be documented in form of a C comment for
> people writing background workers and for those who might want to "fix" this
> in the future.

Well, I suppose we could do that.  Would we then add the same comment
to worker_spi, which does the same thing for the same reason, and
every future contrib module that does stuff with background workers
which we might accept?

It might be better to document this in bgworker.sgml instead.  That
already documents some facts about exiting:

  
   If bgw_restart_time for a background worker is
   configured as BGW_NEVER_RESTART, or if it exits with an exit
   code of 0 or is terminated by TerminateBackgroundWorker,
   it will be automatically unregistered by the postmaster on exit.
   Otherwise, it will be restarted after the time period configured via
   bgw_restart_time, or immediately if the postmaster
   reinitializes the cluster due to a backend failure.  Backends which need
   to suspend execution only temporarily should use an interruptible sleep
   rather than exiting; this can be achieved by calling
   WaitLatch(). Make sure the
   WL_POSTMASTER_DEATH flag is set when calling that function, and
   verify the return code for a prompt exit in the emergency case that
   postgres itself has terminated.
  

That paragraph leaves out a number of important details, like:

1. A background worker that exits with any exit code other than 0 or 1
will cause a postmaster crash-and-restart cycle.

2. Therefore, an exit of code 1 should be used whenever the process
wants to be restarted in accordance with bgw_restart_time, and is
therefore in some sense the "normal" way for a background worker to
exit.

3. The aforementioned details about how 9.3 behavior was different
from current behavior.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Piotr Stefaniak]

On 2016-06-29 18:58, Robert Haas wrote:

This code predates be7558162acc5578d0b2cf0c8d4c76b6076ce352, prior to
which proc_exit(0) forced an immediate, unconditional restart.  It's
true that, given that commit, changing this code to do proc_exit(0)
instead of proc_exit(1) would be harmless.  However, people writing
background workers that might need to work with 9.3 would be best
advised to stick with proc_exit(1).  Therefore, I maintain that this
is not broken and doesn't need to be fixed.


Then I'd argue that it ought to be documented in form of a C comment for 
people writing background workers and for those who might want to "fix" 
this in the future.




--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Robert Haas]

On Mon, Jun 27, 2016 at 11:40 PM, Michael Paquier
 wrote:
> On Tue, Jun 28, 2016 at 6:49 AM, Robert Haas  wrote:
>> On Sun, Jun 26, 2016 at 6:19 AM, Piotr Stefaniak
>>  wrote:
 while investigating the shm_mq code and its testing module I made some
 cosmetic improvements there. You can see them in the attached diff file.
>>>
>>> Revised patch attached.
>>
>> The first hunk of this corrects an outdated comment, so we should
>> certainly apply that.  I'm not seeing what the value of the other bits
>> is.
>
> - proc_exit(1);
> + proc_exit(0);
> Looking again at this thread with fresh eyes, isn't the origin of the
> confusion the fact that we do need to have a non-zero error code so as
> the worker is never restarted thanks to BGW_NEVER_RESTART? Even with
> that, it is a strange concept to leave with proc_exit(1) in the case
> where a worker left correctly..

This code predates be7558162acc5578d0b2cf0c8d4c76b6076ce352, prior to
which proc_exit(0) forced an immediate, unconditional restart.  It's
true that, given that commit, changing this code to do proc_exit(0)
instead of proc_exit(1) would be harmless.  However, people writing
background workers that might need to work with 9.3 would be best
advised to stick with proc_exit(1).  Therefore, I maintain that this
is not broken and doesn't need to be fixed.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Michael Paquier]

On Tue, Jun 28, 2016 at 6:49 AM, Robert Haas  wrote:
> On Sun, Jun 26, 2016 at 6:19 AM, Piotr Stefaniak
>  wrote:
>>> while investigating the shm_mq code and its testing module I made some
>>> cosmetic improvements there. You can see them in the attached diff file.
>>
>> Revised patch attached.
>
> The first hunk of this corrects an outdated comment, so we should
> certainly apply that.  I'm not seeing what the value of the other bits
> is.

- proc_exit(1);
+ proc_exit(0);
Looking again at this thread with fresh eyes, isn't the origin of the
confusion the fact that we do need to have a non-zero error code so as
the worker is never restarted thanks to BGW_NEVER_RESTART? Even with
that, it is a strange concept to leave with proc_exit(1) in the case
where a worker left correctly..
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Robert Haas]

On Sun, Jun 26, 2016 at 6:19 AM, Piotr Stefaniak
 wrote:
>> while investigating the shm_mq code and its testing module I made some
>> cosmetic improvements there. You can see them in the attached diff file.
>
> Revised patch attached.

The first hunk of this corrects an outdated comment, so we should
certainly apply that.  I'm not seeing what the value of the other bits
is.

-- 
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Piotr Stefaniak]

On 2016-05-16 21:40, Piotr Stefaniak wrote:

Hello,

while investigating the shm_mq code and its testing module I made some
cosmetic improvements there. You can see them in the attached diff file.


Revised patch attached.

commit 3ff1afa84e4bc22f153c876e2f0588327a8a004e
Author: Piotr Stefaniak 
Date:   Thu Apr 28 18:36:16 2016 +0200

Cosmetic improvements around shm_mq and test_shm_mq.

diff --git a/src/backend/storage/ipc/shm_mq.c b/src/backend/storage/ipc/shm_mq.c
index cc1f04d..64a5475 100644
--- a/src/backend/storage/ipc/shm_mq.c
+++ b/src/backend/storage/ipc/shm_mq.c
@@ -103,7 +103,7 @@ struct shm_mq
  * locally by copying the chunks into a backend-local buffer.  mqh_buffer is
  * the buffer, and mqh_buflen is the number of bytes allocated for it.
  *
- * mqh_partial_message_bytes, mqh_expected_bytes, and mqh_length_word_complete
+ * mqh_partial_bytes, mqh_expected_bytes, and mqh_length_word_complete
  * are used to track the state of non-blocking operations.  When the caller
  * attempts a non-blocking operation that returns SHM_MQ_WOULD_BLOCK, they
  * are expected to retry the call at a later time with the same argument;
diff --git a/src/test/modules/test_shm_mq/setup.c b/src/test/modules/test_shm_mq/setup.c
index 5bd2820..a0f3962 100644
--- a/src/test/modules/test_shm_mq/setup.c
+++ b/src/test/modules/test_shm_mq/setup.c
@@ -125,7 +125,7 @@ setup_dynamic_shared_memory(int64 queue_size, int nworkers,
 	segsize = shm_toc_estimate();
 
 	/* Create the shared memory segment and establish a table of contents. */
-	seg = dsm_create(shm_toc_estimate(), 0);
+	seg = dsm_create(segsize, 0);
 	toc = shm_toc_create(PG_TEST_SHM_MQ_MAGIC, dsm_segment_address(seg),
 		 segsize);
 
diff --git a/src/test/modules/test_shm_mq/worker.c b/src/test/modules/test_shm_mq/worker.c
index 638649b..a94414a 100644
--- a/src/test/modules/test_shm_mq/worker.c
+++ b/src/test/modules/test_shm_mq/worker.c
@@ -139,7 +139,7 @@ test_shm_mq_main(Datum main_arg)
 	 * we can go ahead and exit.
 	 */
 	dsm_detach(seg);
-	proc_exit(1);
+	proc_exit(0);
 }
 
 /*

-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Piotr Stefaniak]

On 2016-05-17 19:05, Tom Lane wrote:

Michael Paquier  writes:

On Tue, May 17, 2016 at 4:40 AM, Piotr Stefaniak
 wrote:
-toc_bytes = offsetof(shm_toc, toc_entry) +nentry * sizeof(shm_toc_entry)
+toc_bytes = offsetof(shm_toc, toc_entry) + nentry * sizeof(shm_toc_entry)
 + allocated_bytes;



I don't recall the exact reason, but this is intentional style
(memories from a patchwork with Tom).


Well, it's not so much intentional as that pgindent will make it look like
that no matter what you do --- it's got some weird interaction with
sizeof, offsetof, and typedef names versus operators later on the same
line.  I'd call that a pgindent bug myself, but have no particular desire
to try to fix it.


From my understanding, it's because pg_bsd_indent thinks that 
"(shm_toc)" is a cast since shm_toc is a keyword (type alias fetched 
from the "list of typedefs" file in this case, to be precise), forcing 
the "+" to be a unary operator instead of binary.


One easy way to work around this bug is putting shm_toc into its own 
parentheses but I'd prefer dropping this particular fix from my 
"cosmetic changes" patch until pg_bsd_indent is fixed.


I may come up with a possible fix for this bug, but don't hold your breath.


--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Tom Lane]

Michael Paquier  writes:
> On Tue, May 17, 2016 at 4:40 AM, Piotr Stefaniak
>  wrote:
> -toc_bytes = offsetof(shm_toc, toc_entry) +nentry * sizeof(shm_toc_entry)
> +toc_bytes = offsetof(shm_toc, toc_entry) + nentry * sizeof(shm_toc_entry)
>  + allocated_bytes;

> I don't recall the exact reason, but this is intentional style
> (memories from a patchwork with Tom).

Well, it's not so much intentional as that pgindent will make it look like
that no matter what you do --- it's got some weird interaction with
sizeof, offsetof, and typedef names versus operators later on the same
line.  I'd call that a pgindent bug myself, but have no particular desire
to try to fix it.

You could possibly make it look nicer by splitting into multiple lines,
but you'll need to try pgindent'ing to see if you've actually improved the
end result at all.

regards, tom lane


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers

Re: [HACKERS] A couple of cosmetic changes around shared memory code

[Michael Paquier]

On Tue, May 17, 2016 at 4:40 AM, Piotr Stefaniak
 wrote:
> while investigating the shm_mq code and its testing module I made some
> cosmetic improvements there. You can see them in the attached diff file.

-toc_bytes = offsetof(shm_toc, toc_entry) +nentry * sizeof(shm_toc_entry)
+toc_bytes = offsetof(shm_toc, toc_entry) + nentry * sizeof(shm_toc_entry)
 + allocated_bytes;
I don't recall the exact reason, but this is intentional style
(memories from a patchwork with Tom). See for example geo_ops.c or
pl_funcs.c. Though it is true that things are not completely
consistent in the code with offset.

-seg = dsm_create(shm_toc_estimate(), 0);
+seg = dsm_create(segsize, 0);
Yep.

-proc_exit(1);
+proc_exit(0);
Agreed here. I don't see why this should not exit with 0 if there have
not been any errors.
-- 
Michael


-- 
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers