Much thanks for your most informative reply, Alex -
But the main point I was trying to make is that this behaviour
is just plain wrong IMHO :
: (struct Ptr '( P W I ) '(1 . 8) '(2 . 2) '(3 . 4))
-> (1 2 3)
: (struct Ptr '( B . 32 ))
-> (1 0 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0)
The '3' value should be at byte 12, not byte 10 .
As you say, it makes no difference if there is NO struct
description, ie. with:
: (struct Ptr NIL '(1 . 8) '(2 . 2) '(3 . 4))
the misalignment of the 3rd member will still be wrong.
It is asking alot of programmers to always look out for alignment
issues and insert missing pad byte items, and then filter them out
from lists returned by 'struct . I think 'struct should handle
insertion of these pad bytes during layout, and skip over them
during extract.
I think at a minimum '(struct )' should be modified to have
more sense to know that if it has just laid out a 2 byte integer
on a 2 byte boundary, and it needs to layout a 4 byte integer,
then it needs to insert 2 pad bytes. This can't be too difficult
to do , given the equation I gave:
O = O + S + ((O&M) ? (A - (O&M)) : 0)
would do this, as shown in previous mail.
And maybe 'struct could be augmented so that when it has
a struct description, it could use it when laying out a
structure in memory, and would not require cons pairs
on each value entry - a cons pair using expected element
size from the description could be generated for each value -
ie . like my 'carsz' function does - then I would have had to write it.
The alignment is the main issue - after that is fixed, it would much easier
to then augment struct to handle user specified alignments .
As for native issues, how would you call this C function in picolisp ? :
struct BF { unsigned char f1:2 , :4, f2:2; };
register struct BF fbf( register struct BF x asm("rcx") )
{...}
To call this function, register 'RCX' must be loaded with a bit field
value, and another such bit field value will be returned in a
register, which is known only to the compiler and in .debug_info .
Eventually, I hope there will be a way of calling such functions in
picolisp (preserving bit-field structure) and of getting 'struct to
honor correct native "C" alignment, and then user specified alignments !
I will investigate what it would take to make this happen ..
Thanks for your tips on my code - most useful!
On 06/12/2021, Alexander Burger <a...@software-lab.de> wrote:
> Hi Jason,
>
> thanks a lot for your long post!
>
>> '(struct ...)' can guess C native structure alignment wrong:
>> ...
>> So, please ammend 'struct' et al to get this
>> alignment right.
>
> I think we have a misunderstanding here. The 'struct' function does not
> "guess"
> anything. It does not even have the slightest idea about how the structure
> on
> the C level looks like.
>
> 'native' and 'struct' are completely generic. It is the programmer who must
> pass
> all information about the sizes, layout and alignments of C data
> structures.
> Otherwise, they would need to parse all involved include files, with
> relevant
> environment definitions and constants properly set up.
>
> All arguments to 'native' and 'struct' are elements of well-defined
> bit-sizes,
> not nebulous types like 'int' which may have a different number of bits
> depending on platform and C-compiler. The same holds for alignments of
> course,
> which may depend on the optimization level the called library was compiled
> in.
>
> Thus, the programmer or some frontend needs to find out these things for
> his
> platform, and pass the right values. And an alignment is nothing more than a
> few
> inserted bytes.
>
> In fact, there is a rudimentary mechanism for this in the PicoLisp release.
> It is the 'sysdefs' function
>
> https://software-lab.de/doc/refS.html#sysdefs
>
> and the related @src/sysdefs.c and @lib/sysdefs files. It is used, for
> example,
> in the system networking library @lib/net.l to acces network structures
> like
> 'sockaddr_in6' in a portable way.
>
> The trick is to pass the task to the system C compiler via @src/sysdefs.c,
> and
> this can also be done for other system-dependent parts. Also, there is
> probably
> no other way, as only a C compiler can preprocess all defines and includes
> to
> determine the right values.
>
>
>> Since PicoLisp seems to be a mainly Linux-centric implementation,
>
> I would not say so. PicoLisp is not specially aware of Linux, it just needs
> a
> POSIX system.
>
>
>> And why does 'struct, when given
>> the full structure description and
>> a list of arguments, insist on
>> those argments being cons pairs
>> of ( val size ) , when it already
>> knows what the size MUST be from
>> the structure description?
>
> Hmm, I'm not sure what you mean. There is no redundancy, and no full
> structure
> description. You can alway access only a part of a larger memory area.
>
> 'struct' expects a "result specification" (second argument), which has no
> total
> size, but is a single atom or (recursively) a list of result
> specifications.
> These may be cons pairs like (N . 4), but the 4 is no size but a count
> (long[4]).
>
> Or are you talking of 'native'? This does indeed take a total size,
> necessary
> before any processing of the argument, to allow allocation of a local
> buffer. Or
> do you want 'native' to traverse the whole list at *runtime* every time,
> just to
> determine the size? So this size must be just large enough. No problem if it
> is
> larger than the structure size.
>
>
>> Also, lack of support for UNSIGNED integers
>> of size < P (pointer) is a pain!
>>
>> Please support 'U (unsigned int) and 'H
>> (unsigned short) atom types !
>
> Unsiged numbers are fairly trivial to convert. Just ADD the base to the
> number
> and and AND it with (base-1).
>
> (de unsigned (N)
> (& `(dec (** 2 32)) (+ N `(** 2 32))) )
>
> This gives, after having been 'read':
>
> (de unsigned (N)
> (& 4294967295 (+ N 4294967296)) )
>
>
> I want to keep some limit to the number of key-symbols supported by
> 'native'.
> Keep in mind that all this is interpreted at runtime, and each new 'case'
> or
> 'if' slows it down!
>
>
>> Also '(native) should be augmented to handle
>> function parameters which are registers,
>
> How do you mean that? Depending on the ABI of the runtime platform, the
> first
> few arguments are *always* passed in registers, the rest on the stack.
>
>
>> Also what about 128-bit integers ( __{,u}int128 ) ?
>> and long doubles?
>
>
> 128-bit integers are also wasteful to be built-in. If really needed, just
> fetch
> two 64-bit values, and SHIFT and OR them.
>
> Tenbyte floats are a valid feature, and more difficult to emulate (though
> not
> impossible, by accessing the mantissa and exponent separately). But they
> are
> normally used for *intermediati* results, not as function arguments or
> structure
> elements (again: alignment problems). So here too, I would not waste
> runtime resources on them.
>
>
>> PicoLisp is so far in advance of most alternatives, I'd really like
>> to continue using it in projects, but unless I can get this
>> structure member alignment issue sorted easily,
>
> Yes, but as I wrote above, this should (and can)not be done by 'native' and
> 'struct'. They are just the internal workhorses to interface to the
> C-libraries. Think about extending the 'sysdef' principles.
>
>
>> Thanks for PicoLisp, which is otherwise excellent, but yet
>> can still be improved !
>
> Very true! This is always possible and necessary :)
>
> ☺/ A!ex
>
>
> P.S.
>
> : (eval (append (list 'struct 'Ptr ''(B . 32)) (C~carsz '(( B . 32 ))
> (need 32 0))))
>
> Off topic, but note that calling 'eval' on a temporarily built list should
> never
> be necessary in such situations. It is a waste to build a list only to
> execute
> it once. More lispy is 'apply' or 'pass'. The above expression would be
> something like:
>
> : (apply struct (C~carsz '((B . 32)) (need 32 0)) Ptr '(B . 32))
>
> --
> UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
>
--
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe
