Re: memory leak doing gsk_environment_open()

[Charles Mills]

Good to know. I like the default behavior, but good to know. Thanks,

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Carl Kugler
Sent: Thursday, February 04, 2016 11:54 AM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: memory leak doing gsk_environment_open()

> if the requested storage that caused the heap to expand gets returned 
> with a free() or delete I do not believe the C runtime issues a 
> STORAGE RELEASE for the additional increment of heap: it keeps it around in 
> anticipation of future malloc() or new requests. It does not RELEASE the heap 
> storage until the enclave ends.

You can control that with the FREE option on the HEAP LE runtime option. 
See z/OS 2.2.0>z/OS Language Environment>z/OS Language Environment Programming 
Reference>Language Environment runtime options>Using the Language Environment 
runtime options>HEAP.
http://www-01.ibm.com/support/knowledgecenter/SSLTBW_2.2.0/com.ibm.zos.v2r2.ceea300/clheap.htm?lang=en-us

"KEEP
Specifies that an increment to user heap storage is not released when the last 
of the storage within that increment is freed.
FREE
Specifies that an increment to user heap storage is released when the last of 
the storage within that increment is freed."

KEEP is the default.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Robert A. Rosenberg]

If the extra heap size is minor and one-time (being reused when 
needed) I would say to just let it stay allocated. Using the setting 
to free it after use might be a bad idea since the next time you need 
it you will go through another allocate cycle again followed by 
another release one. Is the extra effort/processing worth the saving 
of the Heap storage while not in use?




At 11:43 -0800 on 02/04/2016, Charles Mills wrote about Re: memory 
leak doing gsk_environment_open():



OK, I ran some tests.

The bad news is yes, in my environment, GSK seems to leak storage -- 
that is, it allocates storage that it does not free. The good news 
is that (1) it seems to be "one time," not per session; and (2) the 
storage is modest: eight allocations totaling 0x510 bytes.


I see an increase in HEAP (the heap specifically named HEAP) with 
GSK as opposed to plain old TCP, but it is not dramatic -- nowhere 
near a megabyte -- somewhere around 100K.


HTH

Charles

-Original Message-
From: IBM Mainframe Discussion List 
[mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf Of Charles Mills

Sent: Tuesday, February 02, 2016 4:44 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: memory leak doing gsk_environment_open()


   
I use the gsk functions from C++ with no (known!) issues. I will 
look at the code when I get back to the office Thursday.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Janet Graff]

That was an awesome suggestion!  I am having a great time going through my code 
and finding a couple of leaks.  I do note that I have several leaks on 
setlocale().  It's only 828 bytes but since I call it once per thread it might 
add up.

I have another leak on dllinit() for 155CC0 bytes

Does anyone have any suggestions for freeing that memory?

Janet

>Try the following runtime option HEAPCHK(ON,1,0,10,10). It will produce 
>a dump with a trace of all memory allocations that have no free.

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Charles Mills]

OK, I ran some tests.

The bad news is yes, in my environment, GSK seems to leak storage -- that is, 
it allocates storage that it does not free. The good news is that (1) it seems 
to be "one time," not per session; and (2) the storage is modest: eight 
allocations totaling 0x510 bytes.

I see an increase in HEAP (the heap specifically named HEAP) with GSK as 
opposed to plain old TCP, but it is not dramatic -- nowhere near a megabyte -- 
somewhere around 100K.

HTH

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Charles Mills
Sent: Tuesday, February 02, 2016 4:44 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: Re: memory leak doing gsk_environment_open()



I use the gsk functions from C++ with no (known!) issues. I will look at the 
code when I get back to the office Thursday. 

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Rob Schramm]

I am curious if running GSKSVR to take care of cert cache helps.

On Thu, Feb 4, 2016, 2:54 PM Carl Kugler  wrote:

> > if the requested storage that caused the heap to expand gets returned
> with a free() or delete I do not believe the C runtime issues a STORAGE
> > RELEASE for the additional increment of heap: it keeps it around in
> anticipation of future malloc() or new requests. It does not RELEASE the
> > heap storage until the enclave ends.
>
> You can control that with the FREE option on the HEAP LE runtime option.
> See z/OS 2.2.0>z/OS Language Environment>z/OS Language Environment
> Programming Reference>Language Environment runtime options>Using the
> Language Environment runtime options>HEAP.
>
> http://www-01.ibm.com/support/knowledgecenter/SSLTBW_2.2.0/com.ibm.zos.v2r2.ceea300/clheap.htm?lang=en-us
>
> "KEEP
> Specifies that an increment to user heap storage is not released when the
> last of the storage within that increment is freed.
> FREE
> Specifies that an increment to user heap storage is released when the last
> of the storage within that increment is freed."
>
> KEEP is the default.
>
>-Carl
>
> --
> For IBM-MAIN subscribe / signoff / archive access instructions,
> send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN
>
-- 

Rob Schramm
The Art of Mainframe, Inc

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Carl Kugler]

> if the requested storage that caused the heap to expand gets returned with a 
> free() or delete I do not believe the C runtime issues a STORAGE
> RELEASE for the additional increment of heap: it keeps it around in 
> anticipation of future malloc() or new requests. It does not RELEASE the 
> heap storage until the enclave ends.

You can control that with the FREE option on the HEAP LE runtime option. 
See z/OS 2.2.0>z/OS Language Environment>z/OS Language Environment Programming 
Reference>Language Environment runtime options>Using the Language Environment 
runtime options>HEAP.
http://www-01.ibm.com/support/knowledgecenter/SSLTBW_2.2.0/com.ibm.zos.v2r2.ceea300/clheap.htm?lang=en-us

"KEEP
Specifies that an increment to user heap storage is not released when the last 
of the storage within that increment is freed.
FREE
Specifies that an increment to user heap storage is released when the last of 
the storage within that increment is freed."

KEEP is the default.

   -Carl

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Charles Mills]

Janet, I *think* there may be a misunderstanding here. Here is my simplified 
*guess* of how C memory management works. Credit where credit is due: I am 
expanding on a point made by Peter Farley.

When your C program starts up, the runtime uses STORAGE OBTAIN to allocate a 
heap (several heaps, and a stack -- I said this was simplified). The runtime 
uses that heap to satisfy malloc() and new requests. If a request exceeds the 
remaining heap, the runtime issues a STORAGE OBTAIN for more storage. I suspect 
that is the 1MB increase in allocated storage that you are seeing. (BTW you can 
tune the initial heap size and the additional increment size with #pragma 
statements in your C.)

Now, here is where I think your analysis may be going wrong: if the requested 
storage that caused the heap to expand gets returned with a free() or delete I 
do not believe the C runtime issues a STORAGE RELEASE for the additional 
increment of heap: it keeps it around in anticipation of future malloc() or new 
requests. It does not RELEASE the heap storage until the enclave ends. An 
analysis tool such as Omegamon or TMON will still show the storage as in use -- 
even though it is not really in use. It is in use from z/OS's point of view but 
not from the C library's point of view.

If the allocated storage does not continue to increase (as David suggests) and 
if HEAPCHK does not show "lots" of unreleased storage at the end of your 
program, then I think you are good. Lots? Well, in my experience, exactly two 
unreturned allocations, one of 0x828 and one of 0x40 bytes, is normal. WARNING: 
HEAPCHK may make your program unbelievably slow, slow enough sometimes to 
expose timing issues in the code. My product, which normally completes 
initialization in about five seconds, instead takes about five minutes to come 
up, and nearly as long to come down.

The only way I know of that this "no RELEASE" strategy could be a problem would 
be that the storage is not available for other non-C-library allocations, such 
as VSAM or DB2 buffers or some other non-C program that was sharing the address 
space.

HTH

Charles

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Janet Graff
Sent: Tuesday, February 02, 2016 2:24 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: memory leak doing gsk_environment_open()

I have a long running STC that does an HTTP sequence using secure sockets.  I 
can monitor the memory usage throughout my calls.  I note that my above the 
line storage shows a 1M increase after the gsk_environement_open().  That 
memory is not freed after the gsk_environment_shutdown(), 
gsk_environment_close() or any of the other APIs that close the socket.

Has anyone else noticed this?

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

memory leak doing gsk_environment_open()

[Janet Graff]

I have a long running STC that does an HTTP sequence using secure sockets.  I 
can monitor the memory usage throughout my calls.  I note that my above the 
line storage shows a 1M increase after the gsk_environement_open().  That 
memory is not freed after the gsk_environment_shutdown(), 
gsk_environment_close() or any of the other APIs that close the socket.

Has anyone else noticed this?

Janet

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Janet Graff]

I have run RPTSTG on the application several times in the past and tuned 
things.   I can do this again.  However I can see this leak every time I run 
through an HTTP creation of socket, request, and socket termination.  Which is 
what makes me believe that either the memory isn't cleaned up or the sequence 
of functions I am calling is incorrect.

I see the memory usage on every gsk_environment_open() but I never see the 
memory freed on any of the close or shutdown calls.

This is indeed C.

Janet

 replied to message ---
Janet,

I assume this is a C application by the long and lower-case function names you 
mentioned.  Have you run it with the LE RPTSTG option turned on?  Maybe your 
HEAP parameters cause LE to request that 1M storage as a secondary heap chunk 
because the "gsk_*" routines need some heap storage, and I do not believe that 
heap chunks are freed until termination of the enclave, even if there are no 
current storage allocations in the chunk(s).  RPTSTG output may help you tune 
the initial heap size so that secondary chunks are not needed.

Just a thought.

Peter

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Farley, Peter x23353]

Janet,

I assume this is a C application by the long and lower-case function names you 
mentioned.  Have you run it with the LE RPTSTG option turned on?  Maybe your 
HEAP parameters cause LE to request that 1M storage as a secondary heap chunk 
because the "gsk_*" routines need some heap storage, and I do not believe that 
heap chunks are freed until termination of the enclave, even if there are no 
current storage allocations in the chunk(s).  RPTSTG output may help you tune 
the initial heap size so that secondary chunks are not needed.

Just a thought.

Peter

-Original Message-
From: IBM Mainframe Discussion List [mailto:IBM-MAIN@LISTSERV.UA.EDU] On Behalf 
Of Janet Graff
Sent: Tuesday, February 02, 2016 5:24 PM
To: IBM-MAIN@LISTSERV.UA.EDU
Subject: memory leak doing gsk_environment_open()

I have a long running STC that does an HTTP sequence using secure sockets.  I 
can monitor the memory usage throughout my calls.  I note that my above the 
line storage shows a 1M increase after the gsk_environement_open().  That 
memory is not freed after the gsk_environment_shutdown(), 
gsk_environment_close() or any of the other APIs that close the socket.

Has anyone else noticed this?

Janet

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

This message and any attachments are intended only for the use of the addressee 
and may contain information that is privileged and confidential. If the reader 
of the message is not the intended recipient or an authorized representative of 
the intended recipient, you are hereby notified that any dissemination of this 
communication is strictly prohibited. If you have received this communication 
in error, please notify us immediately by e-mail and delete the message and any 
attachments from your system.


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[David Crayford]

Try the following runtime option HEAPCHK(ON,1,0,10,10). It will produce 
a dump with a trace of all memory allocations that have no free. 
However, it's not unusual for APIs to allocate memory once and store the 
pointer in WSA, kind of like a singleton. That's how the standard stream 
libraries work and it may be the case with system SSL. If memory is not 
increasing every time you call gsk_environment_open() then I would 
suggest it may be working as designed.


On 3/02/2016 7:33 AM, Janet Graff wrote:

I have run RPTSTG on the application several times in the past and tuned 
things.   I can do this again.  However I can see this leak every time I run 
through an HTTP creation of socket, request, and socket termination.  Which is 
what makes me believe that either the memory isn't cleaned up or the sequence 
of functions I am calling is incorrect.

I see the memory usage on every gsk_environment_open() but I never see the 
memory freed on any of the close or shutdown calls.

This is indeed C.

Janet

 replied to message ---
Janet,

I assume this is a C application by the long and lower-case function names you mentioned. 
 Have you run it with the LE RPTSTG option turned on?  Maybe your HEAP parameters cause 
LE to request that 1M storage as a secondary heap chunk because the "gsk_*" 
routines need some heap storage, and I do not believe that heap chunks are freed until 
termination of the enclave, even if there are no current storage allocations in the 
chunk(s).  RPTSTG output may help you tune the initial heap size so that secondary chunks 
are not needed.

Just a thought.

Peter

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN

Re: memory leak doing gsk_environment_open()

[Charles Mills]

I use the gsk functions from C++ with no (known!) issues. I will look at the 
code when I get back to the office Thursday. 


CharlesSent from a mobile; please excuse the brevity

 Original message 
From: Janet Graff <004dc9e91b6d-dmarc-requ...@listserv.ua.edu> 
Date: 02/02/2016  3:33 PM  (GMT-08:00) 
To: IBM-MAIN@LISTSERV.UA.EDU 
Subject: Re: memory leak doing gsk_environment_open() 

I have run RPTSTG on the application several times in the past and tuned 
things.   I can do this again.  However I can see this leak every time I run 
through an HTTP creation of socket, request, and socket termination.  Which is 
what makes me believe that either the memory isn't cleaned up or the sequence 
of functions I am calling is incorrect.

I see the memory usage on every gsk_environment_open() but I never see the 
memory freed on any of the close or shutdown calls.

This is indeed C.

Janet

 replied to message ---
Janet,

I assume this is a C application by the long and lower-case function names you 
mentioned.  Have you run it with the LE RPTSTG option turned on?  Maybe your 
HEAP parameters cause LE to request that 1M storage as a secondary heap chunk 
because the "gsk_*" routines need some heap storage, and I do not believe that 
heap chunks are freed until termination of the enclave, even if there are no 
current storage allocations in the chunk(s).  RPTSTG output may help you tune 
the initial heap size so that secondary chunks are not needed.

Just a thought.

Peter

--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN


--
For IBM-MAIN subscribe / signoff / archive access instructions,
send email to lists...@listserv.ua.edu with the message: INFO IBM-MAIN