Re: [HACKERS] pg_shmem_allocations view

2016-05-09 Thread Michael Paquier
On Fri, May 6, 2016 at 8:01 AM, Andres Freund wrote: > Here's a rebased version. I remember why I didn't call the column > "offset" (as Michael complained about upthread), it's a keyword... Oh, an old patch resurrected from the dead... Reading again the patch +* Increase number of initilize

Re: [HACKERS] pg_shmem_allocations view

2016-05-05 Thread Andres Freund
On 2014-09-19 23:07:07 -0500, Michael Paquier wrote: > On Mon, Aug 18, 2014 at 1:12 PM, Robert Haas wrote: > > On Mon, Aug 18, 2014 at 1:27 PM, Tom Lane wrote: > >> Robert Haas writes: > >>> I thought you were printing actual pointer addresses. If you're just > >>> printing offsets relative to

Re: [HACKERS] pg_shmem_allocations view

2014-09-19 Thread Michael Paquier
On Mon, Aug 18, 2014 at 1:12 PM, Robert Haas wrote: > On Mon, Aug 18, 2014 at 1:27 PM, Tom Lane wrote: >> Robert Haas writes: >>> I thought you were printing actual pointer addresses. If you're just >>> printing offsets relative to wherever the segment happens to be >>> mapped, I don't care abo

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Robert Haas
On Mon, Aug 18, 2014 at 1:27 PM, Tom Lane wrote: > Robert Haas writes: >> I thought you were printing actual pointer addresses. If you're just >> printing offsets relative to wherever the segment happens to be >> mapped, I don't care about that. > > Well, that just means that it's not an *obviou

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Andres Freund
On 2014-08-18 13:27:07 -0400, Tom Lane wrote: > I still like the idea of providing something comparable to > MemoryContextStats, rather than creating a SQL interface. The problem > with a SQL interface is you can't interrogate it unless (1) you are not > already inside a query and (2) the client i

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Tom Lane
Robert Haas writes: > I thought you were printing actual pointer addresses. If you're just > printing offsets relative to wherever the segment happens to be > mapped, I don't care about that. Well, that just means that it's not an *obvious* security risk. I still like the idea of providing some

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Robert Haas
On Mon, Aug 18, 2014 at 12:51 PM, Andres Freund wrote: > On 2014-08-18 12:50:27 -0400, Robert Haas wrote: >> On Mon, Aug 18, 2014 at 12:46 PM, Andres Freund >> wrote: >> > You realize that you can pretty much recompute the offsets from the >> > sizes of the individual allocations anyway? >> >> S

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Tom Lane
Robert Haas writes: > I wouldn't mind having a MemoryContextStats()-type function that could > be used to print this information out by attaching to the backend with > gdb, but the utility of exposing it at the SQL level seems very > marginal to me. +1 for doing it like that.

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Andres Freund
On 2014-08-18 12:50:27 -0400, Robert Haas wrote: > On Mon, Aug 18, 2014 at 12:46 PM, Andres Freund > wrote: > > You realize that you can pretty much recompute the offsets from the > > sizes of the individual allocations anyway? > > Sure, if you know the segment base. Do you? Err? The offset do

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Robert Haas
On Mon, Aug 18, 2014 at 12:46 PM, Andres Freund wrote: > On 2014-08-18 12:41:58 -0400, Robert Haas wrote: >> On Mon, Aug 18, 2014 at 12:00 PM, Andres Freund >> wrote: >> >> Unfortunately, that information also has some security implications. >> >> I'm sure someone trying to exploit any future st

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Andres Freund
On 2014-08-18 12:41:58 -0400, Robert Haas wrote: > On Mon, Aug 18, 2014 at 12:00 PM, Andres Freund > wrote: > >> Unfortunately, that information also has some security implications. > >> I'm sure someone trying to exploit any future stack-overrun > >> vulnerability will be very happy to have more

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Robert Haas
On Mon, Aug 18, 2014 at 12:00 PM, Andres Freund wrote: >> Unfortunately, that information also has some security implications. >> I'm sure someone trying to exploit any future stack-overrun >> vulnerability will be very happy to have more rather than less >> information about the layout of the pro

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Tom Lane
Andres Freund writes: > On 2014-08-18 12:33:44 -0400, Tom Lane wrote: >> Well, we could have the implementation code in those modules but not >> provide any SQL-level access to it without installing an extension. >> The only extra thing visible in the .h files would be a function or two. > That'd

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Andres Freund
On 2014-08-18 12:33:44 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-08-18 12:27:12 -0400, Tom Lane wrote: > >> Should we consider putting it into an extension rather than having > >> it in the core system? That would offer some additional protection > >> for production systems, whic

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Tom Lane
Andres Freund writes: > On 2014-08-18 12:27:12 -0400, Tom Lane wrote: >> Should we consider putting it into an extension rather than having >> it in the core system? That would offer some additional protection >> for production systems, which really shouldn't have much need for >> this IMO. > I'

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Andres Freund
On 2014-08-18 12:27:12 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2014-08-18 11:56:44 -0400, Robert Haas wrote: > >> I fully agree with the idea of exposing the amount of free memory in > >> the shared memory segment (as discussed in other emails); that's > >> critical information. But

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Tom Lane
Andres Freund writes: > On 2014-08-18 11:56:44 -0400, Robert Haas wrote: >> I fully agree with the idea of exposing the amount of free memory in >> the shared memory segment (as discussed in other emails); that's >> critical information. But I think exposing address space layout >> information is

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Andres Freund
On 2014-08-18 11:56:44 -0400, Robert Haas wrote: > On Fri, Aug 15, 2014 at 4:20 AM, Andres Freund wrote: > > On 2014-08-15 11:12:11 +0300, Marti Raudsepp wrote: > >> Hi > >> On Thu, May 8, 2014 at 4:28 PM, Andres Freund > >> wrote: > >> > Ok. A new version of the patches implementing that are >

Re: [HACKERS] pg_shmem_allocations view

2014-08-18 Thread Robert Haas
On Fri, Aug 15, 2014 at 4:20 AM, Andres Freund wrote: > On 2014-08-15 11:12:11 +0300, Marti Raudsepp wrote: >> Hi >> On Thu, May 8, 2014 at 4:28 PM, Andres Freund wrote: >> > Ok. A new version of the patches implementing that are >> > attached. Including a couple of small fixups and docs. The lat

Re: [HACKERS] pg_shmem_allocations view

2014-08-15 Thread Andres Freund
On 2014-08-14 22:16:31 -0700, Michael Paquier wrote: > And here are some comments about patch 2: > - Patch applies with some hunks. > - Some typos are present > s#memory segments..#memory segments. (double dots) > s#NULL#NULL (in the docs as this refers to a value) Will fix. > - Your thoughts abo

Re: [HACKERS] pg_shmem_allocations view

2014-08-15 Thread Andres Freund
On 2014-08-15 11:12:11 +0300, Marti Raudsepp wrote: > Hi > > On Thu, May 8, 2014 at 4:28 PM, Andres Freund wrote: > > Ok. A new version of the patches implementing that are > > attached. Including a couple of small fixups and docs. The latter aren't > > extensive, but that doesn't seem to be warr

Re: [HACKERS] pg_shmem_allocations view

2014-08-15 Thread Marti Raudsepp
Hi On Thu, May 8, 2014 at 4:28 PM, Andres Freund wrote: > Ok. A new version of the patches implementing that are > attached. Including a couple of small fixups and docs. The latter aren't > extensive, but that doesn't seem to be warranted anyway. Is it really actually useful to expose the segmen

Re: [HACKERS] pg_shmem_allocations view

2014-08-14 Thread Michael Paquier
And here are some comments about patch 2: - Patch applies with some hunks. - Some typos are present s#memory segments..#memory segments. (double dots) s#NULL#NULL (in the docs as this refers to a value) - Your thoughts about providing separate patches for each view? What this patch does is straight

Re: [HACKERS] pg_shmem_allocations view

2014-08-07 Thread Michael Paquier
On Thu, May 8, 2014 at 10:28 PM, Andres Freund wrote: > Well, we have to live with it for now :) I just had a look at the first patch and got some comments: 1) Instead of using an assertion here, wouldn't it be better to error out if name is NULL, and truncate the name if it is longer than SHMEM_I

Re: [HACKERS] pg_shmem_allocations view

2014-07-14 Thread Abhijit Menon-Sen
At 2014-07-14 16:48:09 -0400, alvhe...@2ndquadrant.com wrote: > > The attachments are there on the archives, and also on my mbox -- and > unlike Robert, I was not copied. I think this is a problem on > Abhijit's end. Yes, it is. I apologise for the noise. -- Abhijit -- Sent via pgsql-hackers

Re: [HACKERS] pg_shmem_allocations view

2014-07-14 Thread Alvaro Herrera
Robert Haas wrote: > On Mon, Jul 14, 2014 at 6:20 AM, Abhijit Menon-Sen > wrote: > > At 2014-05-08 15:28:22 +0200, and...@2ndquadrant.com wrote: > >> > > Hm. Not sure what you're ACKing here ;). > >> > > >> > The idea of giving the unallocated memory a NULL key. > >> > >> Ok. A new version of the

Re: [HACKERS] pg_shmem_allocations view

2014-07-14 Thread Robert Haas
On Mon, Jul 14, 2014 at 6:20 AM, Abhijit Menon-Sen wrote: > At 2014-05-08 15:28:22 +0200, and...@2ndquadrant.com wrote: >> > > Hm. Not sure what you're ACKing here ;). >> > >> > The idea of giving the unallocated memory a NULL key. >> >> Ok. A new version of the patches implementing that are attac

Re: [HACKERS] pg_shmem_allocations view

2014-07-14 Thread Abhijit Menon-Sen
At 2014-05-08 15:28:22 +0200, and...@2ndquadrant.com wrote: > > > > Hm. Not sure what you're ACKing here ;). > > > > The idea of giving the unallocated memory a NULL key. > > Ok. A new version of the patches implementing that are attached. > Including a couple of small fixups and docs. The latter

Re: [HACKERS] pg_shmem_allocations view

2014-05-08 Thread Andres Freund
On 2014-05-08 07:58:34 -0400, Robert Haas wrote: > On Wed, May 7, 2014 at 5:54 PM, Andres Freund wrote: > > Hm. Not sure what you're ACKing here ;). > > The idea of giving the unallocated memory a NULL key. Ok. A new version of the patches implementing that are attached. Including a couple of sm

Re: [HACKERS] pg_shmem_allocations view

2014-05-08 Thread Robert Haas
On Wed, May 7, 2014 at 5:54 PM, Andres Freund wrote: > On 2014-05-07 17:48:15 -0400, Robert Haas wrote: >> On Tue, May 6, 2014 at 6:09 PM, Andres Freund wrote: >> >> I guess I'd vote for >> >> ditching the allocated column completely and outputting the memory >> >> allocated without ShmemIndex us

Re: [HACKERS] pg_shmem_allocations view

2014-05-07 Thread Andres Freund
On 2014-05-07 17:48:15 -0400, Robert Haas wrote: > On Tue, May 6, 2014 at 6:09 PM, Andres Freund wrote: > >> I guess I'd vote for > >> ditching the allocated column completely and outputting the memory > >> allocated without ShmemIndex using some fixed tag (like "ShmemIndex" > >> or "Bootstrap" or

Re: [HACKERS] pg_shmem_allocations view

2014-05-07 Thread Robert Haas
On Tue, May 6, 2014 at 6:09 PM, Andres Freund wrote: >> I guess I'd vote for >> ditching the allocated column completely and outputting the memory >> allocated without ShmemIndex using some fixed tag (like "ShmemIndex" >> or "Bootstrap" or "Overhead" or something). > > My way feels slightly cleane

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Andres Freund
On 2014-05-06 22:04:04 +0100, Simon Riggs wrote: > On 6 May 2014 20:44, Tom Lane wrote: > > Robert Haas writes: > >> On Tue, May 6, 2014 at 2:34 PM, Tom Lane wrote: > >>> FWIW, I vote for fixing (a) now but holding (b) for 9.5. > > > >> I guess I'll vote for applying both. I don't see a lot of

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Andres Freund
On 2014-05-06 15:37:04 -0400, Robert Haas wrote: > On Tue, May 6, 2014 at 2:34 PM, Tom Lane wrote: > > Andres Freund writes: > >> Attached are two patches: > >> a) Patch addin a 'name' parameter to dsm_create(). I think we should > >>apply this to 9.4. > >> b) pg_dynamic_shmem_allocations and

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Simon Riggs
On 6 May 2014 20:44, Tom Lane wrote: > Robert Haas writes: >> On Tue, May 6, 2014 at 2:34 PM, Tom Lane wrote: >>> FWIW, I vote for fixing (a) now but holding (b) for 9.5. > >> I guess I'll vote for applying both. I don't see a lot of risk, and I >> think doing one with out the other is somewhat

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Tom Lane
Robert Haas writes: > On Tue, May 6, 2014 at 2:34 PM, Tom Lane wrote: >> FWIW, I vote for fixing (a) now but holding (b) for 9.5. > I guess I'll vote for applying both. I don't see a lot of risk, and I > think doing one with out the other is somewhat pointless. The difference is that there's n

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Robert Haas
On Tue, May 6, 2014 at 2:34 PM, Tom Lane wrote: > Andres Freund writes: >> Attached are two patches: >> a) Patch addin a 'name' parameter to dsm_create(). I think we should >>apply this to 9.4. >> b) pg_dynamic_shmem_allocations and pg_static_shmem_allocations >>views. The previous versio

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Tom Lane
Andres Freund writes: > Attached are two patches: > a) Patch addin a 'name' parameter to dsm_create(). I think we should >apply this to 9.4. > b) pg_dynamic_shmem_allocations and pg_static_shmem_allocations >views. The previous version didn't include dsm support and didn't >take the re

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Andres Freund
Hi, On 2014-05-06 13:56:41 +0200, Andres Freund wrote: > On 2014-05-05 23:20:43 -0400, Robert Haas wrote: > > On Mon, May 5, 2014 at 6:54 PM, Tom Lane wrote: > > > I'm not confident that it'll be useful either. But I am confident that > > > if we don't put it in now, and decide we want it later,

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Simon Riggs
On 6 May 2014 13:06, Heikki Linnakangas wrote: > The best scheme will depend on how the segments > are used. Best to leave it to the extension author. +1 -- Simon Riggs http://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Training & Services -- Sent via pgsq

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Heikki Linnakangas
On 05/06/2014 02:59 PM, Robert Haas wrote: >Why would we call multiple segments the same thing?? It's not clear to me how someone is going to intelligently name multiple segments used by the same extension. Maybe they'll give them all the same name. Maybe they'll name them all extension_name.p

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Robert Haas
On Tue, May 6, 2014 at 7:45 AM, Simon Riggs wrote: >> The control segment is sized to support a number of dynamic shared >> memory segments not exceeding 64 + 2 *MaxBackends. With default >> settings, that currently works out to 288 segments, or 2306 bytes. >> So, adding a 64-byte name to each of

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Andres Freund
On 2014-05-05 23:20:43 -0400, Robert Haas wrote: > On Mon, May 5, 2014 at 6:54 PM, Tom Lane wrote: > > I'm not confident that it'll be useful either. But I am confident that > > if we don't put it in now, and decide we want it later, there will be > > complaints when we change the API. Better to

Re: [HACKERS] pg_shmem_allocations view

2014-05-06 Thread Simon Riggs
On 5 May 2014 21:54, Robert Haas wrote: > On Mon, May 5, 2014 at 3:09 PM, Tom Lane wrote: >> Robert Haas writes: >>> On Sun, May 4, 2014 at 7:50 AM, Andres Freund >>> wrote: Thinking about this, I think it was a mistake to not add a 'name' field to dynamic shared memory's dsm_control

Re: [HACKERS] pg_shmem_allocations view

2014-05-05 Thread Robert Haas
On Mon, May 5, 2014 at 6:54 PM, Tom Lane wrote: > Robert Haas writes: >> On Mon, May 5, 2014 at 3:09 PM, Tom Lane wrote: >>> And the controlled shared segment is likely to be how big exactly? It's >>> probably not even possible for it to be smaller than a page size, 4K or >>> so depending on th

Re: [HACKERS] pg_shmem_allocations view

2014-05-05 Thread Tom Lane
Robert Haas writes: > On Mon, May 5, 2014 at 3:09 PM, Tom Lane wrote: >> And the controlled shared segment is likely to be how big exactly? It's >> probably not even possible for it to be smaller than a page size, 4K or >> so depending on the OS. I agree with Andres that a name would be a good

Re: [HACKERS] pg_shmem_allocations view

2014-05-05 Thread Robert Haas
On Mon, May 5, 2014 at 3:09 PM, Tom Lane wrote: > Robert Haas writes: >> On Sun, May 4, 2014 at 7:50 AM, Andres Freund wrote: >>> Thinking about this, I think it was a mistake to not add a 'name' field >>> to dynamic shared memory's dsm_control_item. > >> Well, right now a dsm_control_item is 8

Re: [HACKERS] pg_shmem_allocations view

2014-05-05 Thread Andres Freund
On 2014-05-05 15:09:02 -0400, Tom Lane wrote: > > I'm quite in favor of having something like this for the main shared > > memory segment, but I think that's 9.5 material at this point. > > If you're prepared to break the current APIs later to add a name parameter > (which would have to be require

Re: [HACKERS] pg_shmem_allocations view

2014-05-05 Thread Andres Freund
On 2014-05-05 15:04:07 -0400, Robert Haas wrote: > On Sun, May 4, 2014 at 7:50 AM, Andres Freund wrote: > > On 2014-05-04 13:44:17 +0200, Andres Freund wrote: > >> postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC; > >> key | off |size

Re: [HACKERS] pg_shmem_allocations view

2014-05-05 Thread Tom Lane
Robert Haas writes: > On Sun, May 4, 2014 at 7:50 AM, Andres Freund wrote: >> Thinking about this, I think it was a mistake to not add a 'name' field >> to dynamic shared memory's dsm_control_item. > Well, right now a dsm_control_item is 8 bytes. If we add a name field > of our usual 64 bytes,

Re: [HACKERS] pg_shmem_allocations view

2014-05-05 Thread Robert Haas
On Sun, May 4, 2014 at 7:50 AM, Andres Freund wrote: > On 2014-05-04 13:44:17 +0200, Andres Freund wrote: >> postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC; >> key | off |size | allocated >> -+--

Re: [HACKERS] pg_shmem_allocations view

2014-05-04 Thread Euler Taveira
On 04-05-2014 08:44, Andres Freund wrote: > I've more than once wanted to know what allocated shared memory in > postgres installation is used for. Especially with more an more > extensions around that's quite useful. > A few years ago I had to provide such information an did something similar. Is

Re: [HACKERS] pg_shmem_allocations view

2014-05-04 Thread Andres Freund
Hi, On 2014-05-04 13:44:17 +0200, Andres Freund wrote: > postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC; > key | off |size | allocated > -+-+-+--- > Buffer Blocks

Re: [HACKERS] pg_shmem_allocations view

2014-05-04 Thread Andres Freund
Hi, On 2014-05-04 13:44:17 +0200, Andres Freund wrote: > postgres=# SELECT * FROM pg_shmem_allocations ORDER BY size DESC; > key | off |size | allocated > -+-+-+--- > Buffer Blocks