There is external data that determines if a record is no longer needed. The records indicate a request sent to an external system that cannot reply. Periodically the external system is queries as to completed items.
That is my reason for a second area where the incomplete requests are saved while new requests are also recorded. The areas can be switched in a synchronous instruction at which point the previous area is serialized and can be processed. I was hoping for an idea to avoid the data movement if possible while not losing a lot of storage. If I use multiple areas I can keep the incomplete records in their original area and hope that they will be complete by the time I cycle all the areas. Otherwise I will need to copy them. On Mon, 30 Dec 2024 15:11:35 +0100 Bernd Oppolzer <[email protected]> wrote: :>some more questions, sorry ... :> :>if there are (many) records added and removed at the same time (removed :>by scanning the existing records): :>what is the condition that leads to the deletion of existing records? :>Can it be computed on the existing records alone :>or does it depend on the incoming records? Is the a certain ordering on :>the existing or on the incoming records :>(a key sequence) or a matching logic between the incoming and the :>existing records? :> :>If not, why not first delete (and compress) all the existing records :>that can be deleted (on their "own local" conditions); :>and then - in a second step - add the new incoming records at the end of :>the area? :> :>Please explain what I'm still missing ... :> :>Thanks and regards :> :>Bernd :> :> :>Am 30.12.2024 um 14:57 schrieb Binyamin Dissen: :>> 1. Only access (other than INSERT) is the periodic scan. :>> :>> 2. Issue is that additional records are being inserted at the same time. So :>> may mover and move and move again. My thought was a header which had a pointer :>> to the next available area and using a synchronized instruction to take the :>> next slot, :>> :>> 3. Unknown, but probably <1K. Potential large number of records. :>> :>> :>> On Mon, 30 Dec 2024 14:39:54 +0100 Bernd Oppolzer <[email protected]> :>> wrote: :>> :>> :>I have (at least) two or three questions for this requirement, to :>> :>understand it better and to give a better advice: :>> :> :>> :>1. is it necessary to access the individual items directly from outside, :>> :>that is: do they need a pointer or :>> :>a key or an ID or something, or is the sequential scan the only way to :>> :>access them (after the insertion)? :>> :>A pointer is no good solution for that, because it will change, when :>> :>items are moved (see 2.) ... even :>> :>if there is no second area. But there are easy solutions to overcome :>> :>this problem ... if an external "key" :>> :>is really needed. :>> :> :>> :>2. if you are scanning the area sequentially and remove the items that :>> :>are no longer needed, you don't :>> :>need a second area, IMO. You simply move the remaining items "nearer to :>> :>the top", and eliminate the "holes". :>> :> :>> :>3. can you give an upper limit to the size of the items (which then :>> :>gives us the possibility to define a :>> :>reasonable upper limit for the size of the complete area or structure)? :>> :> :>> :>For me, several designs come to mind: :>> :> :>> :>- sort of "records" with length information, like VSAM records in an :>> :>ESDS control interval :>> :> :>> :>- something like DB2 data pages with row ids (fixed size - 4k, 8k, 32k - :>> :>with a row pointer vector :>> :>at the beginning or at the end; this would allow for external IDs :>> :>remaining constant, :>> :>even if the items were moved inside the page) :>> :> :>> :>Kind regards :>> :> :>> :>Bernd :>> :> :>> :> :>> :>Am 29.12.2024 um 21:51 schrieb Binyamin Dissen: :>> :>> The data structure consists of various items of different lengths. The items :>> :>> are added serially and periodically the area is scanned where most items are :>> :>> removed. The remaining items are likely to be removed in a later scan (with :>> :>> new items). :>> :>> :>> :>> My thought was to have two data areas, and during the scan insert the :>> :>> undeleted items into the alternate area. It will be a bit of data moves but is :>> :>> simple. :>> :>> :>> :>> Does Knuth talk about such a data structure? :>> :>> :>> :>> -- :>> :>> Binyamin Dissen <[email protected]> :>> :>> http://www.dissensoftware.com :>> :>> :>> :>> Director, Dissen Software, Bar & Grill - Israel :>> :>> :>> :>> ---------------------------------------------------------------------- :>> :>> For IBM-MAIN subscribe / signoff / archive access instructions, :>> :>> send email to [email protected] with the message: INFO IBM-MAIN :>> :> :>> :>---------------------------------------------------------------------- :>> :>For IBM-MAIN subscribe / signoff / archive access instructions, :>> :>send email to [email protected] with the message: INFO IBM-MAIN :>> :>> -- :>> Binyamin Dissen <[email protected]> :>> http://www.dissensoftware.com :>> :>> Director, Dissen Software, Bar & Grill - Israel :>> :>> ---------------------------------------------------------------------- :>> For IBM-MAIN subscribe / signoff / archive access instructions, :>> send email to [email protected] with the message: INFO IBM-MAIN :> :>---------------------------------------------------------------------- :>For IBM-MAIN subscribe / signoff / archive access instructions, :>send email to [email protected] with the message: INFO IBM-MAIN -- Binyamin Dissen <[email protected]> http://www.dissensoftware.com Director, Dissen Software, Bar & Grill - Israel ---------------------------------------------------------------------- For IBM-MAIN subscribe / signoff / archive access instructions, send email to [email protected] with the message: INFO IBM-MAIN
