None here, Ed. Thanks, Chris On Mon, Aug 31, 2015 at 10:15 AM, Ed <ej...@hotmail.com> wrote:
> I’ll be pleased to write some tests for it that are basically what you > have below. I’d also like to tidy up the lldf branch as it currently is by > rebasing it to get rid of the merge commit. > > Chris, kmx, any objections? > > Ed > > *From:* Chris Marshall <devel.chm...@gmail.com> > *Sent:* Monday, August 31, 2015 10:58 AM > *To:* pdl-devel@lists.sourceforge.net > *Subject:* Re: [Pdl-devel] LongLong precision (fixed) > > The LongLong to Double conversion problems > appear to have been resolved. I would invite > any PDL developers with systems having lots > of RAM to test large memory operations. > > We need to add some tests to check for correct > operation when the PDLs and index values are > >=2**31. For instance, here is a problem caused > by an int where a PDL_Indx should have been: > > > It crashes for me as does this: > > > > pdl> $PDL::BIGPDL = 1; > > pdl> $x = zeros(byte,1024,1024,2049); > > pdl> $y = $x->copy; > > Segmentation fault (core dumped) > > > > but this is ok: > > > > pdl> $PDL::BIGPDL = 1; > > pdl> $x = zeros(byte,1024,1024,2047); > > pdl> $y = $x->copy; > > pdl> q > > > > so there is a problem for piddles with more > > than 2**31 elements. > > t/bigmem.t anyone? > t/bigmemmap.t anyone? > > > --Chris > > > > On 8/26/2015 17:45, kmx wrote: > > Hi all, > > After some off-list discussion with Chris I have fixed his fix in > longlong-double-fix branch. It seems to work, passes all tests and is more > or less ready for merging to master. > > Travis-CI is all green see > <https://travis-ci.org/PDLPorters/pdl/builds/77348194> > https://travis-ci.org/PDLPorters/pdl/builds/77348194 > > All changes: > https://github.com/PDLPorters/pdl/compare/master...longlong-double-fix#files_bucket > > The main idea is a new type PDL_Anyval capable of keeping values of all > types without loosing any precision. Like this: > > typedef enum { PDL_B, PDL_S, PDL_US, PDL_L, PDL_IND, PDL_LL, PDL_F, PDL_D > } pdl_datatypes; > > typedef struct { > pdl_datatypes type; > union { > PDL_Byte B; > PDL_Short S; > PDL_Ushort U; > PDL_Long L; > PDL_Indx N; > PDL_LongLong Q; > PDL_Float F; > PDL_Double D; > } value; > } PDL_Anyval; > > There are still some questions that I want to discuss here: > > *1/ "broken" C API (not that much, but still)* > > Using PDL_Anyval struct (instead of plain double) has some side effects > concerning C API, namely: > PDL->put_offs(..) > PDL->get_offs(..) > PDL->get(..) > PDL->get_pdl_badvalue(..) > > The relevant declarations in Core structure were not changed: > void (*put_offs)(pdl *pdl,PDL_Indx offs, PDL_Anyval val); > PDL_Anyval (*get_offs)(pdl *pdl,PDL_Indx offs); > PDL_Anyval (*get)(pdl *pdl,PDL_Indx *inds); > PDL_Anyval (*get_pdl_badvalue)(pdl *it); > > however the old code like: > if( PDL->get_pdl_badvalue(a) == 0 ) { ... } > does not work anymore (it does not even compile) from obvious reason. > > The new style should be > double bad_a; > ANYVAL_TO_CTYPE(bad_a, double, PDL->get_pdl_badvalue(a)); > if( bad_a == 0 ) { ... } > > BTW: via http://grep.cpan.me/ I have found only one Zaki's module that is > affected (+of course possibly a lot of code not on CPAN) > > There are also four newly introduced ANYVAL_* macros in pdl.h for handling > PDL_Anyval from C code. > #define ANYVAL_TO_SV(outsv,inany) ... > #define ANYVAL_FROM_CTYPE(outany,avtype,inval) ... > #define ANYVAL_TO_CTYPE(outval,ctype,inany) ... > #define ANYVAL_EQ_ANYVAL(x,y) ... > > Which will be a pain for those who want to write their modules/program so > that it works with old PDL versions (without these macros) as well as with > the new version. The only recommendation I have for this scenario is the > following set of defines that has to be added to the C code for > compatibility with older PDL. > > #if PDL_CORE_VERSION < 12 > #define ANYVAL_TO_SV(outsv,inany) outsv = newSVnv((NV)(inany) > #define ANYVAL_FROM_CTYPE(outany,avtype,inval) outany = > (PDL_Double)(inval) > #define ANYVAL_TO_CTYPE(outval,ctype,inany) outval = (ctype)(inany) > #define ANYVAL_EQ_ANYVAL(x,y) (x == y) > #endif > > *2/ typemap & undef handling* > > I am not quite sure what is the expected behaviour when undef SV argument > has to be converted to PDL_Anyval. Now I convert it into a invalid > PDL_Anyval { type=-1, value.B=0 } which might not be the best idea. > > The old behaviour was always (even for undef) use: > $var = (double)SvNV($arg) > > The difference can be seen for example here: > my $p = sequence(4); > PDL::Core::set_c($p, [1], undef); > print $p > The old behaviour is "undefined ... warning" + setting the value to 0 > (zero). The new behaviour is to croak. > > Of course it would be easy to change croak() to silently setting 0 value > but I want to ask: what is correct/expected/best behaviour in this case? > > We are talking here about a typemap so we do not have any context (e.g. > type of a piddle where PDL_Anyval value will be later assigned) therefore > the obvious suggestion for using badval for undef is not possible as it is > not clear which badval. > > *3/ PDL data type auto-detection* > > Auto-detection (we have a SV and want to decide for the best PDL datatype) > is implemented at couple of places. Based on perl's SV type (int/IV or > double/NV) we decide for PDL_Long (on perl with 64bit int for PDL_LongLong) > or PDL_Double. > > Simplified code works like this: > if (SvIOK(sv)) { > #if IVSIZE == 8 > val = (PDL_LongLong)SvIV(sv) > #else > val = (PDL_Long)SvIV(sv) > #endif > } else if (SvOK(sv)) { > val = (PDL_Double)SvNV(sv) /* tries also to convert strings */ > } else { > // do some magick in undef case > } > > or even simpler: > if (SvIOK(sv)) { > #if IVSIZE == 8 > val = (PDL_LongLong)SvIV(sv) > #else > val = (PDL_Long)SvIV(sv) > #endif > } else { > val = (PDL_Double)SvNV(sv) /* tries also to convert strings + undef > */ > } > > This kind of works but I am open to any comments/improvements. > > BTW: let's have: > $p = pdl( [ 1, 2, 3 ] ); # perl integers > print $p; > Is it correct that type of $p is always Double? although LongLong would be > more than enough? > > > Thanks in advance for any feedback. > > -- > kmx > > > > ------------------------------------------------------------------------------ > > > > _______________________________________________ > pdl-devel mailing > listpdl-devel@lists.sourceforge.nethttps://lists.sourceforge.net/lists/listinfo/pdl-devel > > > ------------------------------ > > ------------------------------------------------------------------------------ > > ------------------------------ > _______________________________________________ > pdl-devel mailing list > pdl-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/pdl-devel > >
------------------------------------------------------------------------------
_______________________________________________ pdl-devel mailing list pdl-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/pdl-devel
