Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Toby Thain wrote: > On 18-Jan-10, at 7:38 PM, Markus Hoenicka wrote: > > >> Markus Hoenicka writes: >> >>> information. This check simply looks spurious to me: >>> >>> if(field->charsetnr == 63) >>> >>> >> It probably isn't as wrong as I thought initially. MySQL gurus, please >> correct me if I'm wrong, but I take the following from the manual: >> >> The purpose of the field->charsetnr==63 is to tell TEXT and BLOB >> fields apart (and their relatives of other sizes). Both are >> represented by a FIELD_TYPE_BLOB (or MYSQL_TYPE_BLOB) type. The former >> is a NULL-terminated string with a character set and a collation, the >> latter is a binary object without character set or collation. There is >> apparently no way to tell these apart other than by their >> field->charsetnr value. >> >> Therefore I tend to think the proper fix is to rearrange the case >> statements in the mysql driver (see the _translate_mysql_type >> function). The VAR_STRING and STRING types should be moved below the >> default: entry to never mark those as binary. >> > > I haven't looked at the code, but that sounds good. > > >> Vikram, do I understand correctly that you don't expect to be able to >> store binary strings in a "VARCHAR(256) COLLATE 'binary'" column? If >> we fix the driver as I suggested above, you'd get back only the part >> of the string up to the first NULL. >> > > Seems like a quite reasonable expectation when fetching using _string > function. > > Yes of course. VARCHAR should be treated as NULL terminated character strings and not binary. As long as libdbi is consistent as to what it thinks a "VARCHAR(256) COLLATE 'binary'" refers to, then I'll be happy. Vikram. -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
On 18-Jan-10, at 7:38 PM, Markus Hoenicka wrote: > Markus Hoenicka writes: >> information. This check simply looks spurious to me: >> >> if(field->charsetnr == 63) >> > > It probably isn't as wrong as I thought initially. MySQL gurus, please > correct me if I'm wrong, but I take the following from the manual: > > The purpose of the field->charsetnr==63 is to tell TEXT and BLOB > fields apart (and their relatives of other sizes). Both are > represented by a FIELD_TYPE_BLOB (or MYSQL_TYPE_BLOB) type. The former > is a NULL-terminated string with a character set and a collation, the > latter is a binary object without character set or collation. There is > apparently no way to tell these apart other than by their > field->charsetnr value. > > Therefore I tend to think the proper fix is to rearrange the case > statements in the mysql driver (see the _translate_mysql_type > function). The VAR_STRING and STRING types should be moved below the > default: entry to never mark those as binary. I haven't looked at the code, but that sounds good. > > Vikram, do I understand correctly that you don't expect to be able to > store binary strings in a "VARCHAR(256) COLLATE 'binary'" column? If > we fix the driver as I suggested above, you'd get back only the part > of the string up to the first NULL. Seems like a quite reasonable expectation when fetching using _string function. --Toby > > regards, > Markus -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Markus Hoenicka writes: > information. This check simply looks spurious to me: > > if(field->charsetnr == 63) > It probably isn't as wrong as I thought initially. MySQL gurus, please correct me if I'm wrong, but I take the following from the manual: The purpose of the field->charsetnr==63 is to tell TEXT and BLOB fields apart (and their relatives of other sizes). Both are represented by a FIELD_TYPE_BLOB (or MYSQL_TYPE_BLOB) type. The former is a NULL-terminated string with a character set and a collation, the latter is a binary object without character set or collation. There is apparently no way to tell these apart other than by their field->charsetnr value. Therefore I tend to think the proper fix is to rearrange the case statements in the mysql driver (see the _translate_mysql_type function). The VAR_STRING and STRING types should be moved below the default: entry to never mark those as binary. Vikram, do I understand correctly that you don't expect to be able to store binary strings in a "VARCHAR(256) COLLATE 'binary'" column? If we fix the driver as I suggested above, you'd get back only the part of the string up to the first NULL. regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Toby Thain was heard to say: > I have discussed this issue with Vikram on #mysql and IMHO it is > wrong in concept for a collation change to affect column type: As it > is not concerning the contents of the field but merely > interpretation. Naturally he didn't expect to have to change his code. > > But then perhaps the driver writer held a different opinion. I totally agree here, collation should not affect column type. I still suspect that MySQL reports the column type correctly, but that the driver uses incorrect or insufficient means to interpret this information. This check simply looks spurious to me: if(field->charsetnr == 63) This looks more to me like a crutch than like an author having a different opinion. Toby, isn't there a more foolproof check available in the MySQL C API? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
On 18-Jan-10, at 3:47 AM, Markus Hoenicka wrote: > Toby Thain was heard to say: > >> This is probably driver dependent. IMHO the first check would be to >> see if the unexpected reversion to binary type is occurring within >> libdbi or in libmysqlclient (or mysqld). >> > > I bet it is. The mysql driver uses some weird check (in > _translate_mysql_type) to find out whether or not a field is binary. > Unless I'm dense the code in question may actually cause other > problems as well, I have discussed this issue with Vikram on #mysql and IMHO it is wrong in concept for a collation change to affect column type: As it is not concerning the contents of the field but merely interpretation. Naturally he didn't expect to have to change his code. But then perhaps the driver writer held a different opinion. > so I'd like to ask everyone with some MySQL > knowledge to have a look at it. Sure. --Toby > Also, the existing check may be > version dependent. Vikram, which MySQL version are you running? > > regards, > Markus > > -- > Markus Hoenicka > http://www.mhoenicka.de > AQ score 38 > > > > -- > > Throughout its 18-year history, RSA Conference consistently > attracts the > world's best and brightest in the field, creating opportunities for > Conference > attendees to learn about information security's most important > issues through > interactions with peers, luminaries and emerging and established > companies. > http://p.sf.net/sfu/rsaconf-dev2dev > ___ > libdbi-users mailing list > libdbi-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/libdbi-users -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
Re: [libdbi-users] dbi_result_get_string_idx fails on VARCHAR COLLATE 'binary'
Toby Thain was heard to say: > This is probably driver dependent. IMHO the first check would be to > see if the unexpected reversion to binary type is occurring within > libdbi or in libmysqlclient (or mysqld). > I bet it is. The mysql driver uses some weird check (in _translate_mysql_type) to find out whether or not a field is binary. Unless I'm dense the code in question may actually cause other problems as well, so I'd like to ask everyone with some MySQL knowledge to have a look at it. Also, the existing check may be version dependent. Vikram, which MySQL version are you running? regards, Markus -- Markus Hoenicka http://www.mhoenicka.de AQ score 38 -- Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev ___ libdbi-users mailing list libdbi-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/libdbi-users
