Re: [Ql-Users] lower bound of common heap allocated space

[pjwitte]

On 23/01/2016 07:41, Wolf wrote:

Hi,

I don't quite agree with Per's explanation, at least not as far as
SMSQ/E is concerned.

Once the memory is allocated, A0 points past the header, as Per said.

So, at -4(a0) you will find chp_flag, not chp_len.

Moreover, chp_flag is used by the system :

Consider how SMSQ/E releases memory reserved on the common heap
(in smsq_mem_rchp_asm)

 movem.l reglist,-(sp)   save volatiles
 moveq   #chp.free,d0
 move.l  d0,chp_ownr(a0) set owner free
 move.l  chp_flag(a0),d0 flag address
 beq.s   mrc_rehpis empty, do nothing
 move.l  d0,a1   set flag...
 st  (a1)...now
*
So what happens here is that, if chp_flag isn't empty, the address
it points to will have its MSB set. If you put "grab" in there, then
address $67726162 will have its MSB set to 1.

OK, that most likely won't be a problem because in most cases that
address doesn't exist. But what if you put something else than "grab"
at chp_flag and that points to somewhere in real memory? Oops...

So your initial way of doing things definitely is the right way.


HTH

Wolfgang


No need to disagree, Wolfgang; a fact is a fact and the fact is I was 
wrong and you are right :)


As Tobias writes further on, the header of a heap is not described in 
the QDOS/SMS Bible, ie it is undocumented and should not be (mis)used.


However, I believe he may be confusing User Heap heaps (blocks 
allocated within a heap by MT.ALLOC(?)/sms.alhp) with general, Common 
Heap blocks. User Heap blocks have this 8 byte header, which is 
described in the document, in which the length and pointer may be 
discarded if not needed (but Ive never used this, because it is so 
poorly explained and documented). Regarding memory blocks reserved on 
the Common Heap with MT.ALCHP/sms.achp, they have the same 16 byte 
header in both QDOS and SMSQ.


In the documentation for SNG's DIY memory toolkit he says:

"System entries use all 16 bytes of the common heap header, but the 
last four bytes are unused if you allocate a block with MT.ALCHP. To 
help DISCARD to identify RESERVEd memory, the DIY Toolkit code stores 
the text 'Bufa' at the end of the common heap header,  as a kind of 
'watermark'."


It may well be that hp_rflag/chp_flag location is not used in QDOS (it 
doesnt appear to be so in Minerva 1.98, at least) but as Wolfgang 
demonstrates, it is used by SMSQ, and so it is a baaad idea to use 
this undocumented area, as Simon has done.


That being said, I thought the 'Q' in SMSQ stood for QL compatible and 
in particular: compatible with Quirky programmers! But this one must 
have slipped past TT, or have been deemed too useful to de-implement 
in SMSQ.


Anyways, apologies for any confusion I may have caused.

Per
___
QL-Users Mailing List

Re: [Ql-Users] lower bound of common heap allocated space

[Wolf]

Hi all,

I've released SMSQmulator v. 2.16.

No bugfixes, just some optimisations and possibiity to set the screen 
update rate.


Have fun!

WOlfgang
___
QL-Users Mailing List

Re: [Ql-Users] lower bound of common heap allocated space

[Marcos Cruz]

En/Je/On 2016-01-23 07:41, Wolf escribió / skribis / wrote :

> Once the memory is allocated, A0 points past the header, as Per said.
> 
> So, at -4(a0) you will find chp_flag, not chp_len.

So the header is "normal", i.e. its offsets are positive.

Thank you for the details. I 've removed the markers anyway. It works
fine.

-- 
Marcos Cruz
http://programandala.net
___
QL-Users Mailing List

Re: [Ql-Users] lower bound of common heap allocated space

[Tobias Fröschle]

All,

Apparently the header of a heap item is a data structure we’re not supposed to 
know what it is like, but some seem (or mean) to know. Apparently, this is also 
different for QDOS and SMSQ/E:

The QDOS Technical guide describes the heap item header a bit differently:

"Each heap item is an area of memory (which is a multiple of 8 bytes long), 
together with a pair of longwords: the first is the length of the heap item, 
while the second is a pointer (relative to itself) to the next heap item in the 
list. The use of relative pointers ensures that heaps may be moved. „

That would look like
ds.lhp_len  ; length of heap item
ds.lhp_nxfree   ; next free item in free list (relative pointer)
hpi:  ds.b heap_data

So, in your case, -4(a) would point to the next free heap item in the free 
list. As the free list entry is not used for /allocated/ heap chunks (only on 
those that are worked into a free list), Mr. Goodwin is free to use it for some 
other purpose - On QDOS. Apparently, on SMSQ/E -4(ax) points to something 
different, and thus looks weird to be changed by applications.
Thus, this code is a good example of why you shouldn’t use undocumented data 
structures: They might change over time :)

Regards,
Tobias


> Am 23.01.2016 um 07:41 schrieb Wolf :
> 
> Hi,
> 
> I don't quite agree with Per's explanation, at least not as far as SMSQ/E is 
> concerned.
> 
> Once the memory is allocated, A0 points past the header, as Per said.
> 
> So, at -4(a0) you will find chp_flag, not chp_len.
> 
> Moreover, chp_flag is used by the system :
> 
> Consider how SMSQ/E releases memory reserved on the common heap
> (in smsq_mem_rchp_asm)
> 
>movem.l reglist,-(sp)   save volatiles
>moveq   #chp.free,d0
>move.l  d0,chp_ownr(a0) set owner free
>move.l  chp_flag(a0),d0 flag address
>beq.s   mrc_rehp   is empty, do nothing
>move.l  d0,a1   set flag...
>st  (a1)   ...now
> *
> So what happens here is that, if chp_flag isn't empty, the address
> it points to will have its MSB set. If you put "grab" in there, then address 
> $67726162 will have its MSB set to 1.
> 
> OK, that most likely won't be a problem because in most cases that address 
> doesn't exist. But what if you put something else than "grab" at chp_flag and 
> that points to somewhere in real memory? Oops...
> 
> So your initial way of doing things definitely is the right way.
> 
> 
> HTH
> 
> Wolfgang
> 
> ___
> QL-Users Mailing List

___
QL-Users Mailing List

Re: [Ql-Users] lower bound of common heap allocated space

[Marcos Cruz]

En/Je/On 2016-01-22 20:57, pjwitte escribió / skribis / wrote :

> Whenever the user requests a block of memory in the Common Heap, the
> request is rounded up to the nearest 8 bytes and a 16 byte header is
> added. The header looks like this:
> 
> chp_len  equ$   longLENgth of space in common heap,
> including header
> chp_drlk equ$0004   longpointer to DRiver LinKage (allocated
> space)
> chp_nxfr equ$0004   longrel pointer (-4) to NeXt FRee space
> (free_space)
> chp_ownr equ$0008   longOWNer job id (-1 if not owned)
> chp_flag equ$000c   longaddress of FLAG byte, set when space
> released
> chp_end  equ$0010
> 
> The address returned in register a0 by the MT.ALCHP/sms.achp call
> points past this header to the start of the usable area, so relative
> to this, chp_len is located at -4(a0).

So those field offsets are used downwards, from the address of chp_len,
which is right below the reserved space.

Thank you for the explanation.

Now I see those keys are in their own file in the SMSQ/E sources. I
didn't have them in my files.

-- 
Marcos Cruz
http://programandala.net
___
QL-Users Mailing List

[Ql-Users] lower bound of common heap allocated space

[Marcos Cruz]

I'm extending SuperForth -- the first version of the library is already
online, http://programandala.net/en.program.sfera.html, also on GitHub.

Unfortunately, the old Forth-83 Standard imposed a 64 KiB memory space
and 16-bit cells, what doesn't make sense for QL, so SuperForth uses
special words to access the QL memory outside its own 64 KiB. But it
lacks words to allocate memory from the common heap, so I'm implementing
it.

I extracted the relevant code from Mark Knight's SystemKit/KnightKit
(1994, 1995, public domain), and adapted it. All the S*BASIC stuff
(parameters passing and checking) is unnecessary:


  include 'inc_macros_asm'
  include 'inc_labels_asm'

  code 'aallocate',aallocate_end  ; ( d -- da ior )

  movem.l a0-a3/d2,-(sp)  ; save the SuperForth registers

  pop_l   d1  ; get the required space

  addq.l  #4,d1   ; add 4 to number of bytes, for marker
  move.l  #-1,d2  ; this job will own the memory
  moveq   #mt_alchp,d0; select mt.alchp, grab some heap memory
  trap#1  ; call it to reserve the memory
  tst.l   d0  ; was there an error?
  bmi.s   aallocate_error ; if yes, error
  move.l  #'grab',(a0); put marker in the area reserved.
  adda.l  #4,a0   ; add 4 to base, avoid overwriting marker
  move.l  a0,d1

  movem.l (sp)+,a0-a3/d2  ; restore the SuperForth registers
  push_l  d1  ; push the address
  moveq   #0,d0   ; signal no error
  push_w  d0  ; push the ior code
  next

aallocate_error
  ; d0 = ior
  movem.l (sp)+,a0-a3/d2  ; restore the SuperForth registers
  push_w  d0  ; push the ior code
  next

aallocate_end equ *



At the moment I prefer to to duplicate code at both possible exit
points, for speed.  Besides, I will remove the "grab" marker, which is
used as an address check before releasing the space, because a Forth
programmer is supposed to know what he's doing anyway :)

But I've seen a similar code by Simon N. Goodwin, in his DIY TK (volume
H, file "memory_asm", keyword `reserve`), where he stores the marker
this way:


   moveq#24,d0  MT.ALCHP
   trap #1  Allocate RAM
   tst.ld0
   bmi.sreturn_fp   Return error code
   move.l   #'Bufa',-4(a0)  Identify this block


This uses the long word below the address of the free space, just
returned by the trap.  My doubt is: Why is that lower space supposed to
be free? It may belong to a previous allocated space, or to a node of
the heap structure. Am I missing something? Is there a "safe gap"
between allocated spaces?  So far I've not found this information in the
low-level documentation I have.  Norman Dumbar includes similar code in
his Programming Series, but of course without the checking marker.

-- 
Marcos Cruz
http://programandala.net
___
QL-Users Mailing List

Re: [Ql-Users] lower bound of common heap allocated space

[Wolf]

Hi,

I don't quite agree with Per's explanation, at least not as far as 
SMSQ/E is concerned.


Once the memory is allocated, A0 points past the header, as Per said.

So, at -4(a0) you will find chp_flag, not chp_len.

Moreover, chp_flag is used by the system :

Consider how SMSQ/E releases memory reserved on the common heap
(in smsq_mem_rchp_asm)

movem.l reglist,-(sp)   save volatiles
moveq   #chp.free,d0
move.l  d0,chp_ownr(a0) set owner free
move.l  chp_flag(a0),d0 flag address
beq.s   mrc_rehpis empty, do nothing
move.l  d0,a1   set flag...
st  (a1)...now
*
So what happens here is that, if chp_flag isn't empty, the address
it points to will have its MSB set. If you put "grab" in there, then 
address $67726162 will have its MSB set to 1.


OK, that most likely won't be a problem because in most cases that 
address doesn't exist. But what if you put something else than "grab" at 
chp_flag and that points to somewhere in real memory? Oops...


So your initial way of doing things definitely is the right way.


HTH

Wolfgang

___
QL-Users Mailing List