On Wed, 2006-10-04 at 14:00 -0500, David Nicol wrote: > On 10/4/06, Martin J. Evans <[EMAIL PROTECTED]> wrote: > > > With DBI/DBD::Oracle all values read from the database are scalars. As > > everyone will know, whether something read from the database is a string > > or a number in Perl purely depends on the context it is used in so: > > internally, there are flags in the SV structure that hint as to what > conversions > have been done on the SV already, for efficiency's sake.
As I suspected. > > I have to admit I don't know how the JSON module knows what is a number > > and what is a string in Perl but I see the same issue with Data::Dumper > > so I presume there must be some way to find out if a perl scalar is a > > number or a string. > > It seems like the pure perl Dumper tests values with a regex and optimizes > to numbers when the stringification matches a number template, while > the XS version checks the flags in the SV structure. OK, I only introduced the Data::Dumper example as it seemed very similar but read on - I over simplified a little. However, it seems strange to me that the pure perl Data::Dumper does something different. If I was using Data::Dumper for data interchange and switched from the pure perl to the XS version (or vice versa) and then called another XS based module with the results - they could be different. > > The problem gets a lot worse for me since I do some arithmetic on values > > pulled from the database before converting them to JSON and this is > > where Perl seems to change them into numbers e.g. > > > I don't want to have to do ($var +0) on all the number fields I pull > > from the database (to turn them into numbers) and neither do I want to > > do a '$var .= ""' (to turn all the fields into strings). > > you might have to do exactly that. "$var" will produce a string version. > Sorry. This seems amazing - I really did not want to do this. I guess what I'm saying is that it would be really useful for a DBD to mark a number as a number so it does not start as a string and mutate to a number when I do arithmetic on it. Most DBDs must know if the column is a number one or not (certainly DBD::ODBC does). > Apparently you can tell the JSON module to make everything strings: > http://search.cpan.org/~makamaka/JSON-1.07/lib/JSON.pm#AUTOCONVERT Here was an over simplification. I moved from JSON to JSON Syck. The JSON module is a purl perl module and does "the right thing" for me for all the cases I tried but I changed to JSON Syck which uses the libsyck library for speed. This explains the difference since JSON Syck is XS and will know the flags in the SV structure and JSON won't. > > As an aside (and probably a perl question rather than a DBI one) does > > anyone know why the type of a scalar changes when you use it on the > > right side of an assignment: > > > > perl -MData::Dumper -le '$a="1"; print Dumper($a); $b += $a; print > > Dumper($a);' > > $VAR1 = '1'; > > $VAR1 = 1; > > The has-been-evaluated-as-a-number flag got set on $a when it was > evaluated as a number; then Dumper, with both available, chose the > number format. That's interesting to know, I'll look into that. I was rather surprised the right hand side of an evaluation changed the type of a variable on the right hand side. > > How does JSON and Data::Dumper know whether Perl thinks something is a > > number or a string? > > inspecting the flags; except pure-perl Dumper apparently uses a > regular expression > to identify numbers. Those are guesses. The source is available for > your inspection. I did look at it briefly but not extensively since it was not my primary problem. Thanks for your insights David, they have increased my understanding although I still feel stuck. I could move back to using the JSON module but I could run into problems with fields like house_name_or_number which I'm now guessing JSON will serialise as a number when it looks like a number and a string when not. If I stick with JSON Syck (which is a lot faster and I need the speed), I'm forced to do +0 on all database fields I know are a number and I don't like this at all. I guess I'm rather surprised to hit this issue without seeing anyone else with it after all this time using DBI. Martin -- Martin J. Evans Easysoft Limited http://www.easysoft.com
