Anybody else disagree?
What's your solution to the issues of time-sensitivity and/or large GC
working sets?
OREXXMan
JCL is the buggy whip of 21st century computing. Stabilize it.
Put Pipelines in the z/OS base. Would you rather process data one
character at a time (Unix/C style), or one record at a time?
IBM has been looking for an HLL for program products; REXX is that language.
On Wed, Aug 22, 2018 at 7:40 AM, Rick McGuire <object.r...@gmail.com> wrote:
> Sorry, not going to happen. Neither trivial to implement or useful. It is
> generally very bad practice to try to "help" the language make the garbage
> collection decisions. It invariably leads to worse performance, not better.
>
> Rick
>
> On Wed, Aug 22, 2018 at 7:25 AM Hobart Spitz <orexx...@gmail.com> wrote:
>
>> Let's make it simple:
>>
>> We add a function called (as a working title) SysStorManage defined as
>> follows:
>>
>> SysStorManage(argument):
>>
>> Argument - a character string consisting of one or more of these
>> characters (in upper or lower case, in any combination).
>>
>> A - requests results reported in an array of values.
>>
>> F - reports the amount of "yet to be used space available before the next
>> GC", i.e. free space.
>>
>> G - forces garbage collection and reports P value (below).
>>
>> P - reports 100*F/T, the percent of free space still available to-be-used
>> .
>>
>> S - requests results reported as a string of one or more blank-delimited
>> numbers. This is the default.
>>
>> T - reports the total amount of storage (used and freed) in the
>> compressible pool. This is fixed from the time the data pool is created.
>>
>> Return value:
>>
>> Per A or S, a sequence of numbers corresponding to the requested values
>> to be reported.
>>
>> The actions are done in order:
>>
>> "GF" causes GC and reports the new P value followed by the now free space
>> value.
>> "FG" reports the free space as it was before doing the GC, followed by
>> the new P value.
>>
>>
>> This allows program(s) to:
>>
>> 1. Gather an estimate(s) of the typical usage of a section(s) of
>> code. Call the function before and after, and report/track the
>> difference.
>> 2. Cause a GC at before entering a time critical point block of code.
>> 3. Make it's own decision on when it wants to force an "early" GC.
>> 4. Something like: if SysStorManage("P") > 65 then call SysStorManage
>> "G"
>> 5. Anything else along these lines.
>>
>> Even someone who doesn't know what they are talking about can see that
>> this is both trivial to implement and eminently useful.
>>
>
>>
>> OREXXMan
>> JCL is the buggy whip of 21st century computing. Stabilize it.
>> Put Pipelines in the z/OS base. Would you rather process data one
>> character at a time (Unix/C style), or one record at a time?
>> IBM has been looking for an HLL for program products; REXX is that
>> language.
>>
>> On Tue, Aug 21, 2018 at 7:25 AM, Rick McGuire <object.r...@gmail.com>
>> wrote:
>>
>>>
>>>
>>> On Tue, Aug 21, 2018 at 7:06 AM Hobart Spitz <orexx...@gmail.com> wrote:
>>>
>>>> My bad for assuming the obvious.
>>>>
>>>> In #1, I was referring the free space yet "to be given out". If I'm
>>>> entering a block that I estimate will need 100K (or 1%) of the total space
>>>> pool, and that number was less, GC would be forced, even tho space was not
>>>> fully exhausted. The program could be reasonably assured to execute
>>>> without running into a GC.
>>>>
>>>
>>> This is not something that is predictable in advance. The best that can
>>> be done is to try to maintain a minimum ratio of live-to-free storage in
>>> the heap. This is a determination that is done after a collection to try to
>>> prevent GC thrashing.
>>>
>>>
>>>>
>>>> In #2, I was referring to the free space recovered so far. Presumably
>>>> one could subtract address of the next uncompressed item from the address
>>>> of the end of the last compressed item. The space in between would be free
>>>> and the GC could be terminated at that point, satisfying the requirements
>>>> of the soon to be executed code.
>>>>
>>>
>>> This makes no sense at all. By this point, the expensive part of the GC
>>> is complete (the marking operation). Terminating the sweep early just
>>> because you have a block of sufficient size would prevent other
>>> optimizations and also force the GC operation to be performed more often.
>>>
>>>
>>>>
>>>> In #4, a little analysis should be able to tell you what variables were
>>>> entirely local. DROPing them could include adding them to a free space
>>>> list, no reference count needed.
>>>>
>>>>
>>>> Seriously, this is really a place where you don't really understand
>>> what you are talking about. There are way more things going on than just
>>> what local variables are local. Object references get passed around,
>>> inserted in collections, etc. The marking operation is the only way to
>>> identify storage that can be reclaimed.
>>>
>>> Rick
>>>
>>>
>>>>
>>>> OREXXMan
>>>> JCL is the buggy whip of 21st century computing. Stabilize it.
>>>> Put Pipelines in the z/OS base. Would you rather process data one
>>>> character at a time (Unix/C style), or one record at a time?
>>>> IBM has been looking for an HLL for program products; REXX is that
>>>> language.
>>>>
>>>> On Tue, Aug 21, 2018 at 5:47 AM, Rick McGuire <object.r...@gmail.com>
>>>> wrote:
>>>>
>>>>> We know how much free space is still available to be given out, but we
>>>>> don't know the total amount of free space there is because it is likely a
>>>>> lot of the previously given out storage is now dead objects. A GC event is
>>>>> a fairly expensive process, so you don't want to do one until you really
>>>>> need to, which also maximizes the amount of storage that gets swept up and
>>>>> also reduces memory fragmentation.
>>>>>
>>>>> Rick
>>>>>
>>>>> On Tue, Aug 21, 2018 at 4:45 AM Hobart Spitz <orexx...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> How could you not know how much free space was left and still be able
>>>>>> to determine if a new request can be satisfied?
>>>>>>
>>>>>> OREXXMan
>>>>>> JCL is the buggy whip of 21st century computing. Stabilize it.
>>>>>> Put Pipelines in the z/OS base. Would you rather process data one
>>>>>> character at a time (Unix/C style), or one record at a time?
>>>>>> IBM has been looking for an HLL for program products; REXX is that
>>>>>> language.
>>>>>>
>>>>>> On Mon, Aug 20, 2018 at 5:52 AM, Rick McGuire <object.r...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Aug 19, 2018 at 8:23 PM Hobart Spitz <orexx...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> It's might be too late in the game to make these kinds of comments,
>>>>>>>> but I'll pose them in case it's not. Failing that, the discussion
>>>>>>>> might be
>>>>>>>> useful as input to future requirement(s). I have not looked into the
>>>>>>>> current code for allocation and garbage collection, so my bad if this
>>>>>>>> is
>>>>>>>> redundant or too far afield.
>>>>>>>>
>>>>>>>> Have one, more, or all of these features, singly or in combination,
>>>>>>>> added:
>>>>>>>>
>>>>>>>> 1. A statement/function/hostcommand/option to force garbage
>>>>>>>> collection if the amount of free space was less than a specified
>>>>>>>> amount
>>>>>>>> and/or percentage. It would be issued before entry into a
>>>>>>>> time-critical
>>>>>>>> section of code. I would propose a default percentage of 50%.
>>>>>>>>
>>>>>>>> Not really possible. The only way to actually know the amount of
>>>>>>> available free space is to perform a garbage collection first, which is
>>>>>>> a
>>>>>>> very expensive operation that we try to avoid. The algorithm does
>>>>>>> include
>>>>>>> heap expansion when the ratio of live to free storage falls below a
>>>>>>> threshold.
>>>>>>>
>>>>>>>>
>>>>>>>> 1.
>>>>>>>> 2. A statement/function/hostcommand/option to stop automatic
>>>>>>>> garbage collection if the amount of space freed was greater than a
>>>>>>>> specified amount and/or percentage. It would be issued before
>>>>>>>> sections of
>>>>>>>> code where short delays can be tolerated, but the long delays
>>>>>>>> associated
>>>>>>>> with full garbage collection would be less so. I again propose a
>>>>>>>> 50%
>>>>>>>> default.
>>>>>>>>
>>>>>>>> Again, not possible. Garbage collections get performed whenever
>>>>>>> we're unable to satisfy an allocation request.
>>>>>>>
>>>>>>>>
>>>>>>>> 1. A modification (possibly applying to certain objects only,
>>>>>>>> e.g. large) that would maintain a linked list of free space elements
>>>>>>>> populated each time an eligible object (e.g. large) is completely
>>>>>>>> DROPped
>>>>>>>> (ref count = 0). A reference count might need to be added to the
>>>>>>>> relevant
>>>>>>>> object types as part of the modification. The feature would allow
>>>>>>>> the
>>>>>>>> garbage collection to terminate quickly if there were enough space
>>>>>>>> in the
>>>>>>>> free-space list. A first-fit algorithm might be appropriate here.
>>>>>>>>
>>>>>>>> There are no reference counts on objects, and in general, reference
>>>>>>> counting garbage collection systems are really not feasible because
>>>>>>> circular references are possible (for example, an array that contains a
>>>>>>> reference to itself).
>>>>>>>
>>>>>>>>
>>>>>>>> 1. The addition of a reference count, or similar, and a
>>>>>>>> linked-list of free-space elements, sorted by address. Elements
>>>>>>>> would be
>>>>>>>> added opportunistically where it is known that the object had no
>>>>>>>> additional
>>>>>>>> usage; e.g. local variables in a PROCEDURE, ::ROUTINE, or ::METHOD
>>>>>>>> not
>>>>>>>> passed to another block. Adjacent free-space elements would be
>>>>>>>> combined as
>>>>>>>> detected.
>>>>>>>>
>>>>>>>> Again, reference counts are not possible. Defection of live objects
>>>>>>> is done by tracing and marking objects from a few root references. The
>>>>>>> free
>>>>>>> storage is "swept up" by scanning memory looking for unmarked objects
>>>>>>> and
>>>>>>> reclaiming that storage. Adjacent dead objects are already recombined
>>>>>>> into
>>>>>>> larger blocks.
>>>>>>>
>>>>>>> Rick
>>>>>>>
>>>>>>>
>>>>>>>> An advantage, in addition to more consistent timing, might the
>>>>>>>> reduction in working set size, page reference, and page modification in
>>>>>>>> paging virtual memory systems. Shortened garbage collection events
>>>>>>>> could
>>>>>>>> enable an operating system to treat the process more favorably, either
>>>>>>>> by
>>>>>>>> policy or of incidentally, than if a full-blown garbage collection were
>>>>>>>> done.
>>>>>>>>
>>>>>>>> The above capabilities might allow greater use of OOREXX in new
>>>>>>>> areas, where timing is critical, and potentially even up to operating
>>>>>>>> system components.
>>>>>>>>
>>>>>>>>
>>>>>>>> OREXXMan
>>>>>>>> JCL is the buggy whip of 21st century computing. Stabilize it.
>>>>>>>> Put Pipelines in the z/OS base. Would you rather process data one
>>>>>>>> character at a time (Unix/C style), or one record at a time?
>>>>>>>> IBM has been looking for an HLL for program products; REXX is that
>>>>>>>> language.
>>>>>>>>
>>>>>>>> On Sun, Aug 19, 2018 at 6:29 PM, Rick McGuire <
>>>>>>>> object.r...@gmail.com> wrote:
>>>>>>>>
>>>>>>>>> After the recent discussion about freeing large array storage, I
>>>>>>>>> decided to take another crack at fixing this problem. My largeobject
>>>>>>>>> sandbox branch is my latest attempt at this. This version is actually
>>>>>>>>> simpler than the current one, which never really fixed the issues.
>>>>>>>>> This
>>>>>>>>> version treats "big" objects differently and will return the memory
>>>>>>>>> to the
>>>>>>>>> system when the object is garbage collected. A "big" object is
>>>>>>>>> currently
>>>>>>>>> defined as greater than 1/2 meg for the 32-bit version and 1 meg for
>>>>>>>>> the
>>>>>>>>> 64-bit version. Debugging this also uncovered a couple of bugs in the
>>>>>>>>> current memory manager (already fixed).
>>>>>>>>>
>>>>>>>>> A big part of the simplification came from removing a bit of
>>>>>>>>> baggage from the original OS/2 code that doesn't really apply any
>>>>>>>>> more.
>>>>>>>>> Removing that, allowed a lot of additional code to be deleted that
>>>>>>>>> probably
>>>>>>>>> was never getting used any way. I'm fairly happy with this version,
>>>>>>>>> though
>>>>>>>>> I might tweak some of the tuning parameters a little as I throw some
>>>>>>>>> additional tests at it.
>>>>>>>>>
>>>>>>>>> Rick
>>>>>>>>>
>>>>>>>>> ------------------------------------------------------------
>>>>>>>>> ------------------
>>>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>>>> _______________________________________________
>>>>>>>>> Oorexx-devel mailing list
>>>>>>>>> Oorexx-devel@lists.sourceforge.net
>>>>>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>>>>>>
>>>>>>>>>
>>>>>>>> ------------------------------------------------------------
>>>>>>>> ------------------
>>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______
>>>>>>>> _________________________________________
>>>>>>>> Oorexx-devel mailing list
>>>>>>>> Oorexx-devel@lists.sourceforge.net
>>>>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>>>>>
>>>>>>>
>>>>>>> ------------------------------------------------------------
>>>>>>> ------------------
>>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>>>> _______________________________________________
>>>>>>> Oorexx-devel mailing list
>>>>>>> Oorexx-devel@lists.sourceforge.net
>>>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>>>>
>>>>>>>
>>>>>> ------------------------------------------------------------
>>>>>> ------------------
>>>>>> Check out the vibrant tech community on one of the world's most
>>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______
>>>>>> _________________________________________
>>>>>> Oorexx-devel mailing list
>>>>>> Oorexx-devel@lists.sourceforge.net
>>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>>>
>>>>>
>>>>> ------------------------------------------------------------
>>>>> ------------------
>>>>> Check out the vibrant tech community on one of the world's most
>>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>>>> _______________________________________________
>>>>> Oorexx-devel mailing list
>>>>> Oorexx-devel@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>>
>>>>>
>>>> ------------------------------------------------------------
>>>> ------------------
>>>> Check out the vibrant tech community on one of the world's most
>>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______
>>>> _________________________________________
>>>> Oorexx-devel mailing list
>>>> Oorexx-devel@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>>
>>>
>>> ------------------------------------------------------------
>>> ------------------
>>> Check out the vibrant tech community on one of the world's most
>>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>>> _______________________________________________
>>> Oorexx-devel mailing list
>>> Oorexx-devel@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>>
>>>
>> ------------------------------------------------------------
>> ------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot______
>> _________________________________________
>> Oorexx-devel mailing list
>> Oorexx-devel@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Oorexx-devel mailing list
> Oorexx-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/oorexx-devel
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Oorexx-devel mailing list
Oorexx-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/oorexx-devel
