Re: [PD-dev] libraries that crash in a double-precision runtime
Am 6. März 2020 22:21:53 MEZ schrieb Lucas Cordiviola : >Hi Thomas, > >The only thing I got from deken (for win64) was >"purest_json[v1.4.3](Windows-amd64-64).dek" > >Pd-double refuses to load it complaining: > >"refusing to load 32bit-float object 'json-encode' into 64bit-float Pd" > >Did you compiled it against a Pd-double? > purest_json is from *another* thomas. also i guess that the deken-name (indicating the precision) is simply wrong. mfg.hft.fsl IOhannes ___ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] libraries that crash in a double-precision runtime
sorry the txt was empty. One more try. see attached. Mensaje telepatico asistido por maquinas. On 3/6/2020 6:21 PM, Lucas Cordiviola wrote: Hi Thomas, The only thing I got from deken (for win64) was "purest_json[v1.4.3](Windows-amd64-64).dek" Pd-double refuses to load it complaining: "refusing to load 32bit-float object 'json-encode' into 64bit-float Pd" Did you compiled it against a Pd-double? In case you need a Pd-double compiled for win64 here is one: The list think this link is spam so I'll try in an attached zip (just a txt with the link). see attached. [1] I'm running a self-hosted instance of "Nextcloud" on an asus eeepc 1GB-ram Intel Atom 32-bit 1.6-Ghz with debian/buster with an ADSL connection. Mensaje telepatico asistido por maquinas. On 3/6/2020 12:33 PM, Thomas Grill wrote: Dear all, i have published binaries on deken for many of my externals covering a lot of platforms and variants (i386/x86_64/arm6/arm7/arm64 on darwin/windows/linux, single and double precision floats). Specifically, binaries for Windows and Pd double precision have been newly added. I am not able to test the externals on all platforms. If you like, please give them a try and let me know should there be crashes or other problems. best, Thomas <> ___ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] libraries that crash in a double-precision runtime
Dear all, i have published binaries on deken for many of my externals covering a lot of platforms and variants (i386/x86_64/arm6/arm7/arm64 on darwin/windows/linux, single and double precision floats). Specifically, binaries for Windows and Pd double precision have been newly added. I am not able to test the externals on all platforms. If you like, please give them a try and let me know should there be crashes or other problems. best, Thomas > Am 03.03.2020 um 14:55 schrieb Thomas Grill : > > Hi all, > thanks to IOhannes for debugging. > I have pushed fixes to https://github.com/g/flext which should make flext > (and flext-based objects) fully compatible with 64-bit DSP, that is, > flext-based project should compile and run (or at least, not crash). > > My question is now how to separate 32-bit DSP from 64-bit DSP binaries. There > doesn't seem a separate file extension, right? > How are two variants handled in deken? > > best, Thomas > >> Am 02.03.2020 um 11:51 schrieb IOhannes m zmölnig : >> >> Signierter PGP-Teil >> On 3/2/20 10:43 AM, IOhannes m zmölnig wrote: >>> On 3/1/20 10:51 PM, IOhannes m zmölnig wrote: On 2/26/20 2:28 PM, IOhannes m zmoelnig wrote: > class_new() *might* return a NULL-pointer (and it will, if the precision > doesn't match), so you have to check for that before you access any > class members. > >>> >>> TL;DR there was a bug in Pd that created a few false positives in my >>> original list. >> >> since all flext externals are crashing, i thought i had a look at those >> specifically. >> it turns out, that flext has it's own use for NULL-classes, as it >> internally uses them to create a persistent (and invisible) "flext >> buffer helper" object. >> >> with mismatched float-sizes, this results in calling `pd_new(NULL)` - >> something that never happens in the ordinary flow, where pd_new() only >> gets called in the newmethod of an objectclass. Pd itself doesn't call >> the newmethod of objectclasses with mismatched float-size, so this is >> usually not a problem. >> >> however, it's easy enough to fix. pd_new() already does a NULL-ptr >> check, but only prints an error and happily proceeds to access it's >> members, leading to the inevitable crash. >> doing an early exit in this case, reduces the list of crashing externals >> to the following eleven (11) packages: >> - creb >> - cyclone >> - hcs >> - moonlib >> - nilwind >> - ossia >> - pddp >> - pdlua >> - pof >> - unauthorized >> - Gem (crash fixed upstream, but no release yet) >> - py (this is a false-positive, as it also crashes if the float-sizes do >> match) >> - context (the crash is triggered by the embedded cyclone) >> >> even though, we can fix pd_new() to accept NULL-classes, flext should >> probably handle this explicitly and not attempt to create the "flext >> buffer helper" object if the class-pointer is invalid. >> >> gfadsmr >> IOhannes >> >> >> > > -- > Thomas Grill > http://g.org -- Thomas Grill http://g.org signature.asc Description: Message signed with OpenPGP ___ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] libraries that crash in a double-precision runtime
On 03.03.20 14:55, Thomas Grill wrote:
> Hi all,
> thanks to IOhannes for debugging.
> I have pushed fixes to https://github.com/g/flext which should make flext
> (and flext-based objects) fully compatible with 64-bit DSP, that is,
> flext-based project should compile and run (or at least, not crash).
>
> My question is now how to separate 32-bit DSP from 64-bit DSP binaries. There
> doesn't seem a separate file extension, right?
right.
there's an ongoing discussion at [900]
the gist is:
## current situation
pd-double uses "class_new64()" instead of "class_new()" (which is
reserved for registering sijngle-precision externals).
if a (single-precision) external calls "class_new()" in a
double-precision runtime, it will spit out a warning and return a
NULL-pointer.
vice versa, if a (double-precision) external calls "class_new64()" in a
single-precision runtime, it will spit out a warning and return a
NULL-pointer.
this is to protect externals from being loaded into a runtime of the
wrong floatsize.
to allow simple compilation of double-precision externals, "class_new"
is redefined (in m_pd.h) to "class_new64" if PD_FLOATSIZE==64.
i personally am convinced that this is a *required* protection of the
runtime.
## TODO
however, people are not entirely happy yet ([900]).
there's basically two classes of problems:
- distribution of packages that contain binaries for both float-sizes
- backward compatibility
### packages with both float-sizes
the current implementation doesn't solve the problem on how to
distribute binaries (e.g. via deken) either side-by-side (multiple
binary files) or in a single phat binary (that containes code for both
precisions).
the former would require a way to distinguish the filenames based on the
precision (that is: a filename-extension schema).
people have proposed something like ".l_amd64-DP", which i
personally find rather ugly. a somewhat nicer schema might be
".float64.l_amd64" (which i would prefer, as it keeps the final
extension intact)
the latter (phat binaries that are compiled for both double- and
single-precision) is theoretically possible, but people have to jump
through a lot of hoops to make it work.
this should, and probably could be made simpler. see [1] for a
suggestion to mangle the entry-point (aka setup() function) of an external.
this could make compilation of phat externals much simpler...iff the
only public functions are the setup-functions.
### backward compatibility
the other problem is, that binaries compiled for double-precision use a
new symbol ("class_new64") which is not present in older versions of Pd
(Pd<<0.51).
This is of course not a problem for double-precision only binaries
(which you don't want to load anyhow in the olde single-precision runtime).
but if the binary contains code for *both* single- abd double-precision,
then you might expect to be able to load the single-precision variant in
legacy Pd's.
i think this can be solved by weak linking (aka: "optional symbols" that
are set to NULL if not available).
i have an experimental branch at [2] where i toyed with this idea today,
and so far it seems to work nicely.
the biggest caveat is, that not all OSs support weak linking, and of
course MicroSoft Windows does not.
so my branch uses a little extra magic to implement a replacement using
GetProcAddress(), but that requires to include , which
somewhat bloats m_pd.h
> How are two variants handled in deken?
deken (the cmdline tool) currently inspects the binary to see whether it
imports the "class_new" symbol (in which case it is a single-precision
binary) resp the "class_new64" symbol (in which case it is a
double-precision binary). if both symbols are imported, then the binary
is multi-precision.
this information is encoded into the filename, e.g. a deken package
"foobar[v1.2.3](Linux-ppc-64)(Linux-ppc-32).dek" contains PowerPC
binaries for both single and double-precision.
the deken-plugin (the downloader/installer) just uses the the filename
to determine whether a given search-result matches it's own
architecture/precision.
gmasdr
IOhannes
[900 ]https://github.com/pure-data/pure-data/issues/900
[1] https://lists.puredata.info/pipermail/pd-dev/2019-12/022214.html
[2] https://git.iem.at/pd/pure-data/-/tree/tests/double-API
signature.asc
Description: OpenPGP digital signature
___
Pd-dev mailing list
Pd-dev@lists.iem.at
https://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] libraries that crash in a double-precision runtime
Hi all, thanks to IOhannes for debugging. I have pushed fixes to https://github.com/g/flext which should make flext (and flext-based objects) fully compatible with 64-bit DSP, that is, flext-based project should compile and run (or at least, not crash). My question is now how to separate 32-bit DSP from 64-bit DSP binaries. There doesn't seem a separate file extension, right? How are two variants handled in deken? best, Thomas > Am 02.03.2020 um 11:51 schrieb IOhannes m zmölnig : > > Signierter PGP-Teil > On 3/2/20 10:43 AM, IOhannes m zmölnig wrote: >> On 3/1/20 10:51 PM, IOhannes m zmölnig wrote: >>> On 2/26/20 2:28 PM, IOhannes m zmoelnig wrote: class_new() *might* return a NULL-pointer (and it will, if the precision doesn't match), so you have to check for that before you access any class members. >>> >> >> TL;DR there was a bug in Pd that created a few false positives in my >> original list. > > since all flext externals are crashing, i thought i had a look at those > specifically. > it turns out, that flext has it's own use for NULL-classes, as it > internally uses them to create a persistent (and invisible) "flext > buffer helper" object. > > with mismatched float-sizes, this results in calling `pd_new(NULL)` - > something that never happens in the ordinary flow, where pd_new() only > gets called in the newmethod of an objectclass. Pd itself doesn't call > the newmethod of objectclasses with mismatched float-size, so this is > usually not a problem. > > however, it's easy enough to fix. pd_new() already does a NULL-ptr > check, but only prints an error and happily proceeds to access it's > members, leading to the inevitable crash. > doing an early exit in this case, reduces the list of crashing externals > to the following eleven (11) packages: > - creb > - cyclone > - hcs > - moonlib > - nilwind > - ossia > - pddp > - pdlua > - pof > - unauthorized > - Gem (crash fixed upstream, but no release yet) > - py (this is a false-positive, as it also crashes if the float-sizes do > match) > - context (the crash is triggered by the embedded cyclone) > > even though, we can fix pd_new() to accept NULL-classes, flext should > probably handle this explicitly and not attempt to create the "flext > buffer helper" object if the class-pointer is invalid. > > gfadsmr > IOhannes > > > -- Thomas Grill http://g.org signature.asc Description: Message signed with OpenPGP ___ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] libraries that crash in a double-precision runtime
On 3/2/20 10:43 AM, IOhannes m zmölnig wrote: > On 3/1/20 10:51 PM, IOhannes m zmölnig wrote: >> On 2/26/20 2:28 PM, IOhannes m zmoelnig wrote: >>> class_new() *might* return a NULL-pointer (and it will, if the precision >>> doesn't match), so you have to check for that before you access any >>> class members. >>> >> > > TL;DR there was a bug in Pd that created a few false positives in my > original list. since all flext externals are crashing, i thought i had a look at those specifically. it turns out, that flext has it's own use for NULL-classes, as it internally uses them to create a persistent (and invisible) "flext buffer helper" object. with mismatched float-sizes, this results in calling `pd_new(NULL)` - something that never happens in the ordinary flow, where pd_new() only gets called in the newmethod of an objectclass. Pd itself doesn't call the newmethod of objectclasses with mismatched float-size, so this is usually not a problem. however, it's easy enough to fix. pd_new() already does a NULL-ptr check, but only prints an error and happily proceeds to access it's members, leading to the inevitable crash. doing an early exit in this case, reduces the list of crashing externals to the following eleven (11) packages: - creb - cyclone - hcs - moonlib - nilwind - ossia - pddp - pdlua - pof - unauthorized - Gem (crash fixed upstream, but no release yet) - py (this is a false-positive, as it also crashes if the float-sizes do match) - context (the crash is triggered by the embedded cyclone) even though, we can fix pd_new() to accept NULL-classes, flext should probably handle this explicitly and not attempt to create the "flext buffer helper" object if the class-pointer is invalid. gfadsmr IOhannes signature.asc Description: OpenPGP digital signature ___ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] libraries that crash in a double-precision runtime (was Re: [PD] double precision merged)
On 3/1/20 10:51 PM, IOhannes m zmölnig wrote: > On 2/26/20 2:28 PM, IOhannes m zmoelnig wrote: >> class_new() *might* return a NULL-pointer (and it will, if the precision >> doesn't match), so you have to check for that before you access any >> class members. >> > TL;DR there was a bug in Pd that created a few false positives in my original list. so while i was accusing externals of not taking care of NULL-pointers properly, the code that would print the nice error message when trying to load an external with the wrong float-size would do exactly the same. fixing this (as in 0cc9869dd in the update/0.50 branch on github), reduces the number of libraries that crash when being loaded by two (2). the false positives were "else" and "iemguts". other libraries (like Gem) still crash, but now they are crashing somewhere else... so the list of crashing packages is now: - Gem - absattr - clk - constantq~ - context - creb - cyclone - hcs - ml.lib - moonlib - nilwind - ossia - pddp - pdlua - pof - pool - py - unauthorized - vasp - vinylcontrol~ - xsample - zconf gfmasdr IOhannes signature.asc Description: OpenPGP digital signature ___ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
Re: [PD-dev] libraries that crash in a double-precision runtime (was Re: [PD] double precision merged)
Hi Iohannes, thanks for bringing up double-precision problems with flext. > > > i downloaded the newest version available on deken for each package that > has a binary available for my platform (Linux/amd64), and attempted to > load the binaries in there with a double-precision Pd. i have not followed the double precision developments lately, and all the flext binaries are traditional single-precision compiles. Before i spend time debugging the issue: Are single-precision externals meant to interoperate with Pd double precision without changes? If not, how are single and double precision externals separately identified? thanks, Thomas -- Thomas Grill http://g.org signature.asc Description: Message signed with OpenPGP ___ Pd-dev mailing list Pd-dev@lists.iem.at https://lists.puredata.info/listinfo/pd-dev
