Re: Improve logging when using Huge Pages

2023-07-06 Thread Michael Paquier
On Fri, Jun 23, 2023 at 01:57:51PM +0900, Michael Paquier wrote: > I could not resist adding two checks in the TAP tests to make sure > that we don't report unknown. Perhaps that's not necessary, but that > would provide coverage in a more broader way, and these are cheap. > > I have run one

Re: Improve logging when using Huge Pages

2023-06-22 Thread Michael Paquier
On Tue, Jun 20, 2023 at 06:44:20PM -0500, Justin Pryzby wrote: > On Tue, Jun 13, 2023 at 02:50:30PM +0900, Michael Paquier wrote: >> On Mon, Jun 12, 2023 at 02:37:15PM -0700, Nathan Bossart wrote: >> > Fair enough. I know I've been waffling in the GUC versus function >> > discussion, but FWIW v7

Re: Improve logging when using Huge Pages

2023-06-20 Thread Justin Pryzby
On Tue, Jun 13, 2023 at 02:50:30PM +0900, Michael Paquier wrote: > On Mon, Jun 12, 2023 at 02:37:15PM -0700, Nathan Bossart wrote: > > Fair enough. I know I've been waffling in the GUC versus function > > discussion, but FWIW v7 of the patch looks reasonable to me. > > +

Re: Improve logging when using Huge Pages

2023-06-13 Thread Michael Paquier
On Tue, Jun 13, 2023 at 02:50:30PM +0900, Michael Paquier wrote: > + Assert(strcmp("unknown", GetConfigOption("huge_pages_status", false, > false)) != 0); > > Not sure that there is anything to gain with this assertion in > CreateSharedMemoryAndSemaphores(), because this is pretty much

Re: Improve logging when using Huge Pages

2023-06-13 Thread Michael Paquier
On Mon, Jun 12, 2023 at 11:11:14PM -0700, Nathan Bossart wrote: > On Tue, Jun 13, 2023 at 02:50:30PM +0900, Michael Paquier wrote: >> Don't we need to update save_backend_variables() and add an entry >> in BackendParameters to make other processes launched by EXEC_BACKEND >> inherit the status

Re: Improve logging when using Huge Pages

2023-06-13 Thread Nathan Bossart
On Tue, Jun 13, 2023 at 02:50:30PM +0900, Michael Paquier wrote: > Don't we need to update save_backend_variables() and add an entry > in BackendParameters to make other processes launched by EXEC_BACKEND > inherit the status value set? I thought this was handled by

Re: Improve logging when using Huge Pages

2023-06-12 Thread Michael Paquier
On Mon, Jun 12, 2023 at 02:37:15PM -0700, Nathan Bossart wrote: > Fair enough. I know I've been waffling in the GUC versus function > discussion, but FWIW v7 of the patch looks reasonable to me. + Assert(strcmp("unknown", GetConfigOption("huge_pages_status", false, false)) != 0); Not

Re: Improve logging when using Huge Pages

2023-06-12 Thread Nathan Bossart
On Tue, May 02, 2023 at 11:17:50AM +0900, Michael Paquier wrote: > On Thu, Apr 20, 2023 at 02:16:17PM -0700, Nathan Bossart wrote: >> AFAICT this would involve adding a bool to BackendParameters and using it >> in save_backend_variables() and restore_backend_variables(), which is an >> additional

Re: Improve logging when using Huge Pages

2023-05-01 Thread Michael Paquier
On Thu, Apr 20, 2023 at 02:16:17PM -0700, Nathan Bossart wrote: > AFAICT this would involve adding a bool to BackendParameters and using it > in save_backend_variables() and restore_backend_variables(), which is an > additional 3 lines of code. That doesn't sound too bad to me, but perhaps > I am

Re: Improve logging when using Huge Pages

2023-04-20 Thread Nathan Bossart
On Thu, Mar 23, 2023 at 07:23:28AM +0900, Michael Paquier wrote: > On Wed, Mar 22, 2023 at 05:18:28PM -0500, Justin Pryzby wrote: >> Wow, good point. I think to make it work we'd need put >> huge_pages_active into BackendParameters and handle it in >> save_backend_variables(). If so, that'd be

Re: Improve logging when using Huge Pages

2023-04-11 Thread Kyotaro Horiguchi
At Tue, 11 Apr 2023 15:17:46 +0900, Michael Paquier wrote in > On Thu, Mar 23, 2023 at 05:25:46PM +0900, Kyotaro Horiguchi wrote: > > Wouldn't storing the value in the shared memory itself work? Though, > > that space does become almost dead for the server's lifetime... > > Sure, it would

Re: Improve logging when using Huge Pages

2023-04-11 Thread Michael Paquier
On Thu, Mar 23, 2023 at 05:25:46PM +0900, Kyotaro Horiguchi wrote: > Wouldn't storing the value in the shared memory itself work? Though, > that space does become almost dead for the server's lifetime... Sure, it would work. However, we'd still need an interface for the extra function. At this

Re: Improve logging when using Huge Pages

2023-04-06 Thread Justin Pryzby
On Tue, Mar 28, 2023 at 09:35:30AM +0900, Michael Paquier wrote: > On Thu, Mar 23, 2023 at 08:50:50PM -0500, Justin Pryzby wrote: > > You mean that you abused of it in some custom branch running the CI on > github? If I may ask, what did you do to make sure that huge pages > are set when

Re: Improve logging when using Huge Pages

2023-04-05 Thread Michael Paquier
On Tue, Mar 28, 2023 at 09:35:30AM +0900, Michael Paquier wrote: > The patch needs to provide more details about the unknown state, IMO, > to make it crystal-clear to the users what are the limitations of this > GUC, especially regarding the fact that this is useful when "try"-ing > to allocate

Re: Improve logging when using Huge Pages

2023-03-27 Thread Michael Paquier
On Thu, Mar 23, 2023 at 08:50:50PM -0500, Justin Pryzby wrote: > I'm sure it's possible, but it's also not worth writing a special > implementation just to handle huge_pages_active, which is better written > in 30 lines than in 300 lines. > > If we needed to avoid using a GUC, maybe it'd work to

Re: Improve logging when using Huge Pages

2023-03-23 Thread Justin Pryzby
On Thu, Mar 23, 2023 at 05:25:46PM +0900, Kyotaro Horiguchi wrote: > Wouldn't storing the value in the shared memory itself work? Though, > that space does become almost dead for the server's lifetime... I'm sure it's possible, but it's also not worth writing a special implementation just to

Re: Improve logging when using Huge Pages

2023-03-23 Thread Kyotaro Horiguchi
At Thu, 23 Mar 2023 07:23:28 +0900, Michael Paquier wrote in > On Wed, Mar 22, 2023 at 05:18:28PM -0500, Justin Pryzby wrote: > > Wow, good point. I think to make it work we'd need put > > huge_pages_active into BackendParameters and handle it in > > save_backend_variables(). If so, that'd be

Re: Improve logging when using Huge Pages

2023-03-22 Thread Michael Paquier
On Wed, Mar 22, 2023 at 05:18:28PM -0500, Justin Pryzby wrote: > Wow, good point. I think to make it work we'd need put > huge_pages_active into BackendParameters and handle it in > save_backend_variables(). If so, that'd be strong argument for using a > GUC, which already has all the necessary

Re: Improve logging when using Huge Pages

2023-03-22 Thread Justin Pryzby
On Wed, Mar 22, 2023 at 04:37:01PM +0900, Michael Paquier wrote: > I have noted something.. For the WIN32 case, we have that: > > +++ b/src/backend/port/win32_shmem.c > @@ -327,6 +327,8 @@ retry: > Sleep(1000); > continue; > } > + > + huge_pages_active =

Re: Improve logging when using Huge Pages

2023-03-22 Thread Michael Paquier
On Tue, Mar 21, 2023 at 09:19:41PM -0500, Justin Pryzby wrote: > You set this patch to "waiting on author" twice. Would you let me know > what I could do to help progress the patch? Right now, I have no idea. My mistake, I've been looking at an incorrect version of the patch. Thanks for

Re: Improve logging when using Huge Pages

2023-03-21 Thread Justin Pryzby
On Mon, Mar 20, 2023 at 02:03:13PM +0900, Michael Paquier wrote: > On Mon, Mar 20, 2023 at 01:54:46PM +0900, Michael Paquier wrote: > > The main advantage of a read-only GUC over a function is that users > > would not need to start a postmaster to know if huge pages would be > > active or not.

Re: Improve logging when using Huge Pages

2023-03-19 Thread Michael Paquier
On Mon, Mar 20, 2023 at 01:09:09AM -0400, Tom Lane wrote: > I'm confused here, because Horiguchi-san is saying that that > won't work. I've not checked the code lately, but I think that > "postgres -C var" prints its results before actually attempting > to establish shared memory, so I suspect

Re: Improve logging when using Huge Pages

2023-03-19 Thread Tom Lane
Michael Paquier writes: > On Tue, Mar 14, 2023 at 02:02:19PM +0900, Kyotaro Horiguchi wrote: >> I slightly prefer using a function for this, as if GUC is used, it can >> only return "unknown" for the command "postgres -C >> huge_page_active". However, apart from this advantage, I prefer using >>

Re: Improve logging when using Huge Pages

2023-03-19 Thread Michael Paquier
On Mon, Mar 20, 2023 at 01:54:46PM +0900, Michael Paquier wrote: > The main advantage of a read-only GUC over a function is that users > would not need to start a postmaster to know if huge pages would be > active or not. This is the main reason why a GUC would be a better > fit, in my opinion,

Re: Improve logging when using Huge Pages

2023-03-19 Thread Michael Paquier
On Tue, Mar 14, 2023 at 02:02:19PM +0900, Kyotaro Horiguchi wrote: > Regarding huge_page_active, its value remains constant throughout a > postmaster's lifespan. In this regard, GUC may be a better fit for > this information. The issue with using GUC for this value is that the > postgres command

Re: Improve logging when using Huge Pages

2023-03-13 Thread Kyotaro Horiguchi
At Mon, 13 Mar 2023 21:33:31 +0100, Stephen Frost wrote in > > On Thu, Mar 09, 2023 at 03:02:29PM -0500, Stephen Frost wrote: > > > * Justin Pryzby (pry...@telsasoft.com) wrote: > > > Is there an agreement to use a function, instead ? > > > > Alvaro was -1 on using a function > > > I don’t

Re: Improve logging when using Huge Pages

2023-03-13 Thread Stephen Frost
Greetings, On Mon, Mar 13, 2023 at 21:03 Justin Pryzby wrote: > On Thu, Mar 09, 2023 at 03:02:29PM -0500, Stephen Frost wrote: > > * Justin Pryzby (pry...@telsasoft.com) wrote: > > > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > > > > * Nathan Bossart

Re: Improve logging when using Huge Pages

2023-03-13 Thread Justin Pryzby
On Thu, Mar 09, 2023 at 03:02:29PM -0500, Stephen Frost wrote: > * Justin Pryzby (pry...@telsasoft.com) wrote: > > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > > > * Nathan Bossart (nathandboss...@gmail.com) wrote: > > > > On Wed, Feb 15, 2023 at 10:13:17AM -0800, Nathan

Re: Improve logging when using Huge Pages

2023-03-09 Thread Stephen Frost
Greetings, * Alvaro Herrera (alvhe...@alvh.no-ip.org) wrote: > On 2023-Mar-09, Justin Pryzby wrote: > > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > > > > +Reports whether huge pages are in use by the current instance. > > > > +See for more information. > > >

Re: Improve logging when using Huge Pages

2023-03-09 Thread Stephen Frost
Greetings, * Justin Pryzby (pry...@telsasoft.com) wrote: > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > > * Nathan Bossart (nathandboss...@gmail.com) wrote: > > > On Wed, Feb 15, 2023 at 10:13:17AM -0800, Nathan Bossart wrote: > > > > On Tue, Feb 14, 2023 at 07:32:56PM -0600,

Re: Improve logging when using Huge Pages

2023-03-09 Thread Nathan Bossart
On Thu, Mar 09, 2023 at 06:51:21PM +0100, Alvaro Herrera wrote: > I'd rather make all these other places use "instance" instead. We used > to consider these terms interchangeable, but since we introduced the > glossary to unify the terminology, they are no longer supposed to be. > A server (== a

Re: Improve logging when using Huge Pages

2023-03-09 Thread Nathan Bossart
On Thu, Mar 09, 2023 at 10:38:56AM -0600, Justin Pryzby wrote: > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: >> To that end- perhaps this isn't appropriate as a GUC at all? Why not >> have this just be a system information function? Functionally it really >> provides the same

Re: Improve logging when using Huge Pages

2023-03-09 Thread Alvaro Herrera
On 2023-Mar-09, Justin Pryzby wrote: > On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > > > +Reports whether huge pages are in use by the current instance. > > > +See for more information. > > > > > > I still think we should say "server" in place of "current

Re: Improve logging when using Huge Pages

2023-03-09 Thread Justin Pryzby
On Thu, Mar 09, 2023 at 09:34:10AM -0500, Stephen Frost wrote: > Greetings, > > * Nathan Bossart (nathandboss...@gmail.com) wrote: > > On Wed, Feb 15, 2023 at 10:13:17AM -0800, Nathan Bossart wrote: > > > On Tue, Feb 14, 2023 at 07:32:56PM -0600, Justin Pryzby wrote: > > >> On Mon, Feb 13, 2023

Re: Improve logging when using Huge Pages

2023-03-09 Thread Stephen Frost
Greetings, * Nathan Bossart (nathandboss...@gmail.com) wrote: > On Wed, Feb 15, 2023 at 10:13:17AM -0800, Nathan Bossart wrote: > > On Tue, Feb 14, 2023 at 07:32:56PM -0600, Justin Pryzby wrote: > >> On Mon, Feb 13, 2023 at 08:18:52PM -0800, Nathan Bossart wrote: > >>> I'm curious why you chose

Re: Improve logging when using Huge Pages

2023-03-08 Thread Nathan Bossart
On Wed, Feb 15, 2023 at 10:13:17AM -0800, Nathan Bossart wrote: > On Tue, Feb 14, 2023 at 07:32:56PM -0600, Justin Pryzby wrote: >> On Mon, Feb 13, 2023 at 08:18:52PM -0800, Nathan Bossart wrote: >>> I'm curious why you chose to make this a string instead of an enum. There >>> might be little

Re: Improve logging when using Huge Pages

2023-02-15 Thread Nathan Bossart
On Tue, Feb 14, 2023 at 07:32:56PM -0600, Justin Pryzby wrote: > On Mon, Feb 13, 2023 at 08:18:52PM -0800, Nathan Bossart wrote: >> On Mon, Feb 13, 2023 at 05:22:45PM -0600, Justin Pryzby wrote: >> nitpick: Does this need to be initialized here? > > None of the GUCs' C vars need to be

Re: Improve logging when using Huge Pages

2023-02-14 Thread Justin Pryzby
On Mon, Feb 13, 2023 at 08:18:52PM -0800, Nathan Bossart wrote: > On Mon, Feb 13, 2023 at 05:22:45PM -0600, Justin Pryzby wrote: > > +Reports whether huge pages are in use by the current process. > > +See for more information. > > nitpick: Should this say "server" instead of

Re: Improve logging when using Huge Pages

2023-02-13 Thread Nathan Bossart
On Mon, Feb 13, 2023 at 05:22:45PM -0600, Justin Pryzby wrote: > +Reports whether huge pages are in use by the current process. > +See for more information. nitpick: Should this say "server" instead of "current process"? > +static char *huge_pages_active = "unknown"; /*

Re: Improve logging when using Huge Pages

2023-02-13 Thread Justin Pryzby
On Thu, Feb 09, 2023 at 11:29:09AM -0800, Nathan Bossart wrote: > On Thu, Feb 09, 2023 at 10:40:13AM +0100, Alvaro Herrera wrote: > > On 2023-Feb-08, Justin Pryzby wrote: > >> I don't think it makes sense to run postgres -C huge_pages_active, > >> however, so I see no issue that that would always

Re: Improve logging when using Huge Pages

2023-02-09 Thread Nathan Bossart
On Thu, Feb 09, 2023 at 10:40:13AM +0100, Alvaro Herrera wrote: > On 2023-Feb-08, Justin Pryzby wrote: >> I don't think it makes sense to run postgres -C huge_pages_active, >> however, so I see no issue that that would always returns "false". > > Hmm, I would initialize it to return "unknown"

Re: Improve logging when using Huge Pages

2023-02-09 Thread Alvaro Herrera
On 2023-Feb-08, Justin Pryzby wrote: > I don't think it makes sense to run postgres -C huge_pages_active, > however, so I see no issue that that would always returns "false". Hmm, I would initialize it to return "unknown" rather than "off" — and make sure it turns "off" at the appropriate time.

Re: Improve logging when using Huge Pages

2023-02-08 Thread Justin Pryzby
On Thu, Feb 02, 2023 at 04:53:37PM +0100, Alvaro Herrera wrote: > On 2023-Jan-24, Justin Pryzby wrote: > > On Mon, Jan 23, 2023 at 05:33:35PM -0800, Andres Freund wrote: > > > I'm ok with this being a GUC, it doesn't feel elegant, but I suspect > > > there's > > > no realistic elegant answer. >

Re: Improve logging when using Huge Pages

2023-02-08 Thread Nathan Bossart
On Thu, Feb 02, 2023 at 04:53:37PM +0100, Alvaro Herrera wrote: > Maybe I misread the code (actually I only read the patch), but -- does > it set active=true when huge_pages=on? I think the code only works when > huge_pages=try. That might be pretty confusing; I think it should say > "on" in

Re: Improve logging when using Huge Pages

2023-02-02 Thread Alvaro Herrera
On 2023-Jan-24, Justin Pryzby wrote: > On Mon, Jan 23, 2023 at 05:33:35PM -0800, Andres Freund wrote: > > I'm ok with this being a GUC, it doesn't feel elegant, but I suspect there's > > no realistic elegant answer. > Whether it's a GUC or a function, I propose to name it hugepages_active. > If

Re: Improve logging when using Huge Pages

2023-01-30 Thread Justin Pryzby
On Tue, Jan 24, 2023 at 07:37:29PM -0600, Justin Pryzby wrote: > BTW, I didn't realize it when I made the suggestion, nor when I wrote > the patch, but a GUC was implemented in the v2 patch. >

Re: Improve logging when using Huge Pages

2023-01-24 Thread Justin Pryzby
On Mon, Jan 23, 2023 at 05:33:35PM -0800, Andres Freund wrote: > Hi, > > On 2023-01-23 19:21:00 -0600, Justin Pryzby wrote: > > Michael seemed to support this idea and nobody verbalized hatred of it, > > so I implemented it. In v15, we have shared_memory_size_in_huge_pages, > > so this adds

Re: Improve logging when using Huge Pages

2023-01-23 Thread Andres Freund
Hi, On 2023-01-23 19:21:00 -0600, Justin Pryzby wrote: > Michael seemed to support this idea and nobody verbalized hatred of it, > so I implemented it. In v15, we have shared_memory_size_in_huge_pages, > so this adds effective_huge_pages. > > Feel free to suggest a better name. Hm. Should this

Re: Improve logging when using Huge Pages

2023-01-23 Thread Justin Pryzby
On Wed, Nov 09, 2022 at 02:04:00PM +0900, Michael Paquier wrote: > On Wed, Nov 09, 2022 at 11:47:57AM +0900, Kyotaro Horiguchi wrote: > > Honestly I don't come up with other users of the new > > log-level. Another possible issue is it might be a bit hard for people > > to connect that level to

Re: Improve logging when using Huge Pages

2022-11-08 Thread Michael Paquier
On Wed, Nov 09, 2022 at 11:47:57AM +0900, Kyotaro Horiguchi wrote: > Honestly I don't come up with other users of the new > log-level. Another possible issue is it might be a bit hard for people > to connect that level to huge_pages=try, whereas I think we shouldn't > put a description about the

Re: Improve logging when using Huge Pages

2022-11-08 Thread Kyotaro Horiguchi
At Tue, 8 Nov 2022 11:34:53 +1300, Thomas Munro wrote in > On Tue, Nov 8, 2022 at 4:56 AM Tom Lane wrote: > > Andres Freund writes: > > > On 2022-11-06 14:04:29 +0700, John Naylor wrote: > > Agreed --- changing "on" to be exactly like "try" isn't an improvement. > > If you want "try"

Re: Improve logging when using Huge Pages

2022-11-07 Thread Thomas Munro
On Tue, Nov 8, 2022 at 4:56 AM Tom Lane wrote: > Andres Freund writes: > > On 2022-11-06 14:04:29 +0700, John Naylor wrote: > >> I think the best thing to do is change huge_pages='on' to log a WARNING and > >> fallback to regular pages if the mapping fails. > > > I am strongly opposed to making

Re: Improve logging when using Huge Pages

2022-11-07 Thread Tom Lane
Andres Freund writes: > On 2022-11-06 14:04:29 +0700, John Naylor wrote: >> I think the best thing to do is change huge_pages='on' to log a WARNING and >> fallback to regular pages if the mapping fails. > I am strongly opposed to making 'on' fall back to not using huge pages. That's > what 'try'

Re: Improve logging when using Huge Pages

2022-11-07 Thread Andres Freund
Hi, On 2022-11-06 14:04:29 +0700, John Naylor wrote: > I think the best thing to do is change huge_pages='on' to log a WARNING and > fallback to regular pages if the mapping fails. That way, both dev and prod > can keep the same settings, since 'on' will have both visibility and > robustness. I

Re: Improve logging when using Huge Pages

2022-11-06 Thread Justin Pryzby
On Sun, Nov 06, 2022 at 02:04:29PM +0700, John Naylor wrote: > On Thu, Nov 3, 2022 at 8:11 AM Thomas Munro wrote: > > > > I wonder if the problem here is that people are reluctant to add noise > > to every starting system. There are people who have not configured > > their system and don't want

Re: Improve logging when using Huge Pages

2022-11-06 Thread John Naylor
On Thu, Nov 3, 2022 at 8:11 AM Thomas Munro wrote: > > I wonder if the problem here is that people are reluctant to add noise > to every starting system. There are people who have not configured > their system and don't want to see that noise, and then some people > have configured their system

RE: Improve logging when using Huge Pages

2022-11-04 Thread Shinoda, Noriyoshi (PN Japan FSIP)
; Julien Rouhaud ; Tom Lane ; Kyotaro Horiguchi Subject: Re: Improve logging when using Huge Pages On Wed, Aug 3, 2022 at 8:42 PM Shinoda, Noriyoshi (PN Japan FSIP) wrote: > > As discussed in [1], we're taking this opportunity to return some patchsets > > that don't appear to be ge

Re: Improve logging when using Huge Pages

2022-11-02 Thread Thomas Munro
On Wed, Aug 3, 2022 at 8:42 PM Shinoda, Noriyoshi (PN Japan FSIP) wrote: > > As discussed in [1], we're taking this opportunity to return some patchsets > > that don't appear to be getting enough reviewer interest. > Thank you for your helpful comments. > As you say, my patch doesn't seem to be

RE: Improve logging when using Huge Pages

2022-08-03 Thread Shinoda, Noriyoshi (PN Japan FSIP)
m Lane ; Kyotaro Horiguchi Subject: Re: Improve logging when using Huge Pages As discussed in [1], we're taking this opportunity to return some patchsets that don't appear to be getting enough reviewer interest. This is not a rejection, since we don't necessarily think there's anything unacceptable

Re: Improve logging when using Huge Pages

2022-08-01 Thread Jacob Champion
As discussed in [1], we're taking this opportunity to return some patchsets that don't appear to be getting enough reviewer interest. This is not a rejection, since we don't necessarily think there's anything unacceptable about the entry, but it differs from a standard "Returned with Feedback" in

RE: Improve logging when using Huge Pages

2021-11-21 Thread Shinoda, Noriyoshi (PN Japan FSIP)
; Kyotaro Horiguchi Subject: Re: Improve logging when using Huge Pages On Tue, Nov 2, 2021 at 1:24 AM Fujii Masao wrote: > > > > On 2021/10/29 7:05, Justin Pryzby wrote: > > Hi, > > > > On Wed, Oct 27, 2021 at 06:39:46AM +, Shinoda, Noriyoshi (PN Japan > >

Re: Improve logging when using Huge Pages

2021-11-10 Thread Masahiko Sawada
On Tue, Nov 2, 2021 at 1:24 AM Fujii Masao wrote: > > > > On 2021/10/29 7:05, Justin Pryzby wrote: > > Hi, > > > > On Wed, Oct 27, 2021 at 06:39:46AM +, Shinoda, Noriyoshi (PN Japan > > FSIP) wrote: > >> Thank you for your comment. > >> The attached patch stops message splitting. > >> This

RE: Improve logging when using Huge Pages

2021-11-08 Thread Shinoda, Noriyoshi (PN Japan FSIP)
...@oss.nttdata.com] Sent: Tuesday, November 2, 2021 11:35 PM To: Shinoda, Noriyoshi (PN Japan FSIP) ; pgsql-hack...@postgresql.org; Masahiko Sawada Cc: rjuju...@gmail.com; t...@sss.pgh.pa.us; Kyotaro Horiguchi ; Justin Pryzby Subject: Re: Improve logging when using Huge Pages On 2021/11/02

Re: Improve logging when using Huge Pages

2021-11-02 Thread Fujii Masao
On 2021/11/02 18:31, Shinoda, Noriyoshi (PN Japan FSIP) wrote: Fujii-san, Sawada-san, Thank you for your comment. Also, with the patch, the log message is emitted also during initdb and starting up in single user mode: Certainly the log output when executing the initdb command was a

RE: Improve logging when using Huge Pages

2021-11-02 Thread Shinoda, Noriyoshi (PN Japan FSIP)
rjuju...@gmail.com; t...@sss.pgh.pa.us; Kyotaro Horiguchi ; Justin Pryzby Subject: Re: Improve logging when using Huge Pages On 2021/10/29 16:00, Masahiko Sawada wrote: > Which is noisy. Perhaps it's better to log it only when > IsPostmasterEnvironment is true. +1 Regards, -- Fujii Ma

Re: Improve logging when using Huge Pages

2021-11-01 Thread Fujii Masao
On 2021/10/29 16:00, Masahiko Sawada wrote: Which is noisy. Perhaps it's better to log it only when IsPostmasterEnvironment is true. +1 Regards, -- Fujii Masao Advanced Computing Technology Center Research and Development Headquarters NTT DATA CORPORATION

Re: Improve logging when using Huge Pages

2021-11-01 Thread Fujii Masao
On 2021/10/29 7:05, Justin Pryzby wrote: Hi, On Wed, Oct 27, 2021 at 06:39:46AM +, Shinoda, Noriyoshi (PN Japan FSIP) wrote: Thank you for your comment. The attached patch stops message splitting. This patch also limits the timing of message output when huge_pages = try and HugePages

Re: Improve logging when using Huge Pages

2021-10-29 Thread Masahiko Sawada
On Wed, Oct 27, 2021 at 3:40 PM Shinoda, Noriyoshi (PN Japan FSIP) wrote: > > Hi, > Thank you for your comment. > The attached patch stops message splitting. > This patch also limits the timing of message output when huge_pages = try and > HugePages is not used. > I've looked at the patch. Here

Re: Improve logging when using Huge Pages

2021-10-28 Thread Justin Pryzby
Hi, On Wed, Oct 27, 2021 at 06:39:46AM +, Shinoda, Noriyoshi (PN Japan FSIP) wrote: > Thank you for your comment. > The attached patch stops message splitting. > This patch also limits the timing of message output when huge_pages = try and > HugePages is not used. Thanks for updating the

RE: Improve logging when using Huge Pages

2021-10-27 Thread Shinoda, Noriyoshi (PN Japan FSIP)
, October 22, 2021 11:38 AM To: Shinoda, Noriyoshi (PN Japan FSIP) Cc: masao.fu...@oss.nttdata.com; pgsql-hack...@postgresql.org; rjuju...@gmail.com; t...@sss.pgh.pa.us; Kyotaro Horiguchi Subject: Re: Improve logging when using Huge Pages + ereport(LOG, (errmsg("Anonymous s

Re: Improve logging when using Huge Pages

2021-10-21 Thread Justin Pryzby
+ ereport(LOG, (errmsg("Anonymous shared memory was allocated %s huge pages.", with_hugepages ? "with" : "without"))); You shouldn't break a sentence into pieces like this, since it breaks translation. You don't want an untranslated "without" to appear in the middle of the

RE: Improve logging when using Huge Pages

2021-09-27 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Cc: masao.fu...@oss.nttdata.com; Shinoda, Noriyoshi (PN Japan FSIP) ; pgsql-hack...@postgresql.org; rjuju...@gmail.com; t...@sss.pgh.pa.us Subject: Re: Improve logging when using Huge Pages At Tue, 21 Sep 2021 19:23:22 -0500, Justin Pryzby wrote in > On Wed, Sep 22, 2021 at 02:03:11AM +0900,

Re: Improve logging when using Huge Pages

2021-09-26 Thread Kyotaro Horiguchi
At Tue, 21 Sep 2021 19:23:22 -0500, Justin Pryzby wrote in > On Wed, Sep 22, 2021 at 02:03:11AM +0900, Fujii Masao wrote: > > Another idea is to output "Anonymous shared memory was allocated with > > huge pages" when it's successfully allocated with huge pages, and to output > > "Anonymous

Re: Improve logging when using Huge Pages

2021-09-21 Thread Justin Pryzby
On Wed, Sep 22, 2021 at 02:03:11AM +0900, Fujii Masao wrote: > Another idea is to output "Anonymous shared memory was allocated with > huge pages" when it's successfully allocated with huge pages, and to output > "Anonymous shared memory was allocated without huge pages" > when it's

Re: Improve logging when using Huge Pages

2021-09-21 Thread Fujii Masao
On 2021/09/20 17:55, Shinoda, Noriyoshi (PN Japan FSIP) wrote: I was worried about the same thing as you. So the attached patch gets the value of the kernel parameter vm.nr_hugepages, and if it is the default value of zero, the log level is the same as before. On the other hand, if any value

RE: Improve logging when using Huge Pages

2021-09-20 Thread Shinoda, Noriyoshi (PN Japan FSIP)
ju...@gmail.com; t...@sss.pgh.pa.us Subject: Re: Improve logging when using Huge Pages At Fri, 17 Sep 2021 00:13:41 +0900, Fujii Masao wrote in > > > On 2021/09/08 11:17, Kyotaro Horiguchi wrote: > > I don't dislike the message, but I'm not sure I like the message is > > too

Re: Improve logging when using Huge Pages

2021-09-16 Thread Kyotaro Horiguchi
At Fri, 17 Sep 2021 00:13:41 +0900, Fujii Masao wrote in > > > On 2021/09/08 11:17, Kyotaro Horiguchi wrote: > > I don't dislike the message, but I'm not sure I like the message is > > too verbose, especially about it has DETAILS. It seems to me something > > like the following is sufficient,

Re: Improve logging when using Huge Pages

2021-09-16 Thread Fujii Masao
On 2021/09/08 11:17, Kyotaro Horiguchi wrote: I don't dislike the message, but I'm not sure I like the message is too verbose, especially about it has DETAILS. It seems to me something like the following is sufficient, or I'd like see it even more concise. "fall back anonymous shared memory

Re: Improve logging when using Huge Pages

2021-09-16 Thread Fujii Masao
On 2021/09/07 22:16, Justin Pryzby wrote: On Tue, Sep 07, 2021 at 07:12:36PM +0900, Fujii Masao wrote: One big concern about the patch is that log message is always reported when shared memory fails to be allocated with huge pages enabled when huge_pages=try. Since huge_pages=try is the

Re: Improve logging when using Huge Pages

2021-09-08 Thread Kyotaro Horiguchi
Thanks! At Wed, 8 Sep 2021 07:52:35 +, "Shinoda, Noriyoshi (PN Japan FSIP)" wrote in > Hello, > > Thank you everyone for comments. > I have attached a patch that simply changed the message like the advice from > Horiguchi-san. > > > Even with the patch, there are still some cases where

RE: Improve logging when using Huge Pages

2021-09-08 Thread Shinoda, Noriyoshi (PN Japan FSIP)
m; t...@sss.pgh.pa.us Subject: Re: Improve logging when using Huge Pages At Tue, 7 Sep 2021 08:16:53 -0500, Justin Pryzby wrote in > On Tue, Sep 07, 2021 at 07:12:36PM +0900, Fujii Masao wrote: > > One big concern about the patch is that log message is always > > reported when shared

Re: Improve logging when using Huge Pages

2021-09-07 Thread Kyotaro Horiguchi
At Tue, 7 Sep 2021 08:16:53 -0500, Justin Pryzby wrote in > On Tue, Sep 07, 2021 at 07:12:36PM +0900, Fujii Masao wrote: > > One big concern about the patch is that log message is always reported > > when shared memory fails to be allocated with huge pages enabled > > when huge_pages=try. Since

Re: Improve logging when using Huge Pages

2021-09-07 Thread Justin Pryzby
On Tue, Sep 07, 2021 at 07:12:36PM +0900, Fujii Masao wrote: > One big concern about the patch is that log message is always reported > when shared memory fails to be allocated with huge pages enabled > when huge_pages=try. Since huge_pages=try is the default setting, > many users would see this

Re: Improve logging when using Huge Pages

2021-09-07 Thread Fujii Masao
On 2021/09/07 13:09, Shinoda, Noriyoshi (PN Japan FSIP) wrote: Hello, Thank you everyone for comments. In the thread [1] that Horiguchi told me about, there is already a review going on about GUC for HugePages memory. For this reason, I have removed the new GUC implementation and attached a

RE: Improve logging when using Huge Pages

2021-09-06 Thread Shinoda, Noriyoshi (PN Japan FSIP)
) ; rjuju...@gmail.com; pgsql-hack...@postgresql.org Subject: Re: Improve logging when using Huge Pages On 2021/09/03 23:27, Tom Lane wrote: > Fujii Masao writes: >> IMO, if the level is promoted to LOG, the message should be updated >> so that it follows the error message sty

Re: Improve logging when using Huge Pages

2021-09-03 Thread Fujii Masao
On 2021/09/03 23:27, Tom Lane wrote: Fujii Masao writes: IMO, if the level is promoted to LOG, the message should be updated so that it follows the error message style guide. But I agree that simpler message would be better in this case. So what about something like the following? LOG:

Re: Improve logging when using Huge Pages

2021-09-03 Thread Tom Lane
Fujii Masao writes: > IMO, if the level is promoted to LOG, the message should be updated > so that it follows the error message style guide. But I agree that simpler > message would be better in this case. So what about something like > the following? > LOG: could not map anonymous shared

Re: Improve logging when using Huge Pages

2021-09-03 Thread Fujii Masao
On 2021/09/03 16:49, Kyotaro Horiguchi wrote: IF you are thinking to show that in GUC, you might want to look into the nearby thread [1] Yes, let's discuss this feature at that thread. I have some comment about the patch. - if (huge_pages == HUGE_PAGES_TRY && ptr ==

Re: Improve logging when using Huge Pages

2021-09-03 Thread Kyotaro Horiguchi
Hello. At Fri, 3 Sep 2021 06:28:58 +, "Shinoda, Noriyoshi (PN Japan FSIP)" wrote in > Fujii-san, Julien-san > > Thank you very much for your comment. > I followed your comment and changed the elog function to ereport function and > also changed the log level. The output message is the

RE: Improve logging when using Huge Pages

2021-09-03 Thread Shinoda, Noriyoshi (PN Japan FSIP)
ards, Noriyoshi Shinoda -Original Message- From: Fujii Masao [mailto:masao.fu...@oss.nttdata.com] Sent: Wednesday, September 1, 2021 2:06 AM To: Julien Rouhaud ; Shinoda, Noriyoshi (PN Japan FSIP) Cc: pgsql-hack...@postgresql.org Subject: Re: Improve logging when using Huge Pages On 2

Re: Improve logging when using Huge Pages

2021-08-31 Thread Fujii Masao
On 2021/08/31 15:57, Julien Rouhaud wrote: On Tue, Aug 31, 2021 at 1:37 PM Shinoda, Noriyoshi (PN Japan FSIP) wrote: In the current version, when GUC huge_pages=try, which is the default setting, no log is output regardless of the success or failure of the HugePages acquisition. If you

Re: Improve logging when using Huge Pages

2021-08-31 Thread Julien Rouhaud
On Tue, Aug 31, 2021 at 1:37 PM Shinoda, Noriyoshi (PN Japan FSIP) wrote: > > In the current version, when GUC huge_pages=try, which is the default > setting, no log is output regardless of the success or failure of the > HugePages acquisition. If you want to output logs, you need to set >

Improve logging when using Huge Pages

2021-08-30 Thread Shinoda, Noriyoshi (PN Japan FSIP)
Hi Hackers, In the current version, when GUC huge_pages=try, which is the default setting, no log is output regardless of the success or failure of the HugePages acquisition. If you want to output logs, you need to set log_min_messages=DEBUG3, but it will output a huge amount of extra logs.