Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-21 Thread Tim Bunce
On Tue, Jul 21, 2015 at 01:33:34PM +0100, Martin J. Evans wrote:
 Long, sorry.

No problem. The whole topic is a bit of a mess.

 On 20/07/15 18:00, Tim Bunce wrote:
 On Mon, Jul 20, 2015 at 02:54:53PM +0100, Martin J. Evans wrote:
 On 20/07/15 14:15, Tim Bunce wrote:
 
 I think that would work for me - I'm happy to test it our here if you want 
 to give it a go.
 
 IIRC, when this was last discussed the problem is that some drivers
 might not set DBIc_ROW_COUNT so you can't just use DBIc_ROW_COUNT.
 
 Hence the check that DBIc_ROW_COUNT is not zero. Since the DBI code sets
 it to zero before the call, if it's non-zero after the call we can be
 sure that the driver has set it.
 
 In fact, I just checked, and DBD::ODBC does not seem to call
 DBIc_ROW_COUNT other than to set it to 0 in ODBC.xsi (which is code
 from DBI anyway). Does that sound right?
 
 Nope. Is it setting the underlying structure member directly?
 
 no. All it does is it has a RowCount member in its own imp_sth_st structure 
 which is a SQLLEN (64 bits on 64 bit machines and 32 on 32). Then it:
 
 o dbd_db_execute returns the number of rows or -1 or -2 (error)
   At the end of dbd_st_execute if the affected rows is bigger than INT_MAX 
 and warnings are
   on, it warns the rowcount has been truncated and changes the row count to 
 INT_MAX.

That's reasonable. Hopefully we can do better though.

 o has odbc_st_rows (because it is defined in dbd_xsh.h and I believed
   you needed to implement most of these in the DBD) which casts the
   internal RowCount to an int as odbc_st_rows is defined as returning an int.

The DBI provides a default rows method, in C, that returns DBIc_ROW_COUNT.
So a driver that stores the row count in DBIc_ROW_COUNT doesn't need to
provide a rows method at all (if all it needs to do is return the count).

That translates into not defining the dbd_st_rows macro. If that's not
defined then the rows method in Driver.xst won't get compiled in so
there'll be no call to a driver-provided dbd_st_rows.

 DBD::ODBC also has its own odbc_rows which returns an IV to workaround this 
 issue in DBI when I found it back in 2012.

If DBD::ODBC switched to using DBIc_ROW_COUNT then you could remove
dbd_st_rows/odbc_rows.  (It seems unlikely that sizeof(IV) would ever me
less than sizeof(SQLLEN) but that might be worth an assertion anyway.)


 Looking at 'do' in DBI.pm it just does:
 
 sub do {
   my($dbh, $statement, $attr, @params) = @_;
   my $sth = $dbh-prepare($statement, $attr) or return undef;
   $sth-execute(@params) or return undef;
   my $rows = $sth-rows;
   ($rows == 0) ? 0E0 : $rows;
 }
 
 so doesn't that just end up in dbd_st_rows?

Assuming the driver is using that default do() method, then it'll
end up in dbd_st_rows if the driver has defined a dbd_st_rows macro,
else it'll end up in the DBI's default rows() method.

 If a driver is supposed to set DBIc_ROW_COUNT I'd rather change the
 drivers I maintain to do that especially since in ODBC and 64bit
 SQLRowCount already returns a 64 bit value.
 
 Yeap. That's best.
 
 See above, I don't see how that fits in right now.

Is the only outstanding issue now the 'int' return type on some various
dbd_st_* functions?

 I tried to check my assumptions and this is what I found:
 
 o DBD::ODBC has its own 'do' method because it can use SQLExecDirect instead 
 of prepare/execute. This returns the rows affected correctly as it returns an 
 SV created from the SQLLEN RowCount. So, the do method in DBI (shown above) 
 is neither here nor there for DBD::ODBC.
 
 o DBD::ODBC has a dbd_st_rows which seems to get called if someone calls the 
 rows method.
 dbd_st_rows is defined in dbd_xsh.h as returning an int so this is wrong.

And can simply be removed, per the above.

 o 'execute' or dbd_st_execute returns the rows and is again defined in 
 dbd_xsh as returning an int.
 
 I don't see where DBIc_ROW_COUNT comes in unless you are saying every time a 
 DBD discovers the row count it should call DBIc_ROW_COUNT macro.

DBIc_ROW_COUNT is just a macro for an IV in the imp_sth structure. Most,
if not all, compiled drivers that aren't using DBIc_ROW_COUNT are simply
using their own integer element in the imp_sth structure. In the case of
DBD::Pg that's declared as a plain int type.

So I'd hope and expect a driver can simply use DBIc_ROW_COUNT _instead of_
whatever it's currently using.

 I also noticed something I should have seen before: dbd_st_rows() is
 defined as returning an int. I _think_ it would be safe to change the
 definition to returning an IV since it's only used internally by drivers
 via the Driver.xst template file that does:
 
  XST_mIV(0, dbd_st_rows(sth, imp_sth));
 
 Unless I'm missing something I think that will break most drivers as when I 
 grepped cpan I found most drivers implement dbd_st_rows as:
 
 int dbd_st_rows {
   code
 }

[Sigh] I'm getting a bit rusty at C. I'd forgotten that hurdle.
The int return type affects 

Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-21 Thread Martin J. Evans

Long, sorry.

On 20/07/15 18:00, Tim Bunce wrote:

On Mon, Jul 20, 2015 at 02:54:53PM +0100, Martin J. Evans wrote:

On 20/07/15 14:15, Tim Bunce wrote:

I think that would work for me - I'm happy to test it our here if you want to 
give it a go.

IIRC, when this was last discussed the problem is that some drivers
might not set DBIc_ROW_COUNT so you can't just use DBIc_ROW_COUNT.


Hence the check that DBIc_ROW_COUNT is not zero. Since the DBI code sets
it to zero before the call, if it's non-zero after the call we can be
sure that the driver has set it.


In fact, I just checked, and DBD::ODBC does not seem to call
DBIc_ROW_COUNT other than to set it to 0 in ODBC.xsi (which is code
from DBI anyway). Does that sound right?


Nope. Is it setting the underlying structure member directly?


no. All it does is it has a RowCount member in its own imp_sth_st structure 
which is a SQLLEN (64 bits on 64 bit machines and 32 on 32). Then it:

o dbd_db_execute returns the number of rows or -1 or -2 (error)
  At the end of dbd_st_execute if the affected rows is bigger than INT_MAX and 
warnings are
  on, it warns the rowcount has been truncated and changes the row count to 
INT_MAX.

o has odbc_st_rows (because it is defined in dbd_xsh.h and I believed you 
needed to implement most of these in the DBD) which casts the internal RowCount 
to an int as odbc_st_rows is defined as returning an int.

DBD::ODBC also has its own odbc_rows which returns an IV to workaround this 
issue in DBI when I found it back in 2012.

Note dbd_xsh.h defines dbd_st_rows and dbd_st_execute as returning ints.

Looking at 'do' in DBI.pm it just does:

sub do {
my($dbh, $statement, $attr, @params) = @_;
my $sth = $dbh-prepare($statement, $attr) or return undef;
$sth-execute(@params) or return undef;
my $rows = $sth-rows;
($rows == 0) ? 0E0 : $rows;
}

so doesn't that just end up in dbd_st_rows?


If a driver is supposed to set DBIc_ROW_COUNT I'd rather change the
drivers I maintain to do that especially since in ODBC and 64bit
SQLRowCount already returns a 64 bit value.


Yeap. That's best.


See above, I don't see how that fits in right now.

I tried to check my assumptions and this is what I found:

o DBD::ODBC has its own 'do' method because it can use SQLExecDirect instead of 
prepare/execute. This returns the rows affected correctly as it returns an SV 
created from the SQLLEN RowCount. So, the do method in DBI (shown above) is 
neither here nor there for DBD::ODBC.

o DBD::ODBC has a dbd_st_rows which seems to get called if someone calls the 
rows method.
dbd_st_rows is defined in dbd_xsh.h as returning an int so this is wrong.

o 'execute' or dbd_st_execute returns the rows and is again defined in dbd_xsh 
as returning an int.

I don't see where DBIc_ROW_COUNT comes in unless you are saying every time a 
DBD discovers the row count it should call DBIc_ROW_COUNT macro.


Is there some docs on that or perhaps you could just tell me or point
me at a driver that does it correctly.


No docs, sadly. And I'm not aware of any drivers that do.

I took a look at DBD:Pg and that uses it's own 'rows' structure
member which is defined as an int, and int is used in the code.

I also noticed something I should have seen before: dbd_st_rows() is
defined as returning an int. I _think_ it would be safe to change the
definition to returning an IV since it's only used internally by drivers
via the Driver.xst template file that does:

 XST_mIV(0, dbd_st_rows(sth, imp_sth));


Unless I'm missing something I think that will break most drivers as when I 
grepped cpan I found most drivers implement dbd_st_rows as:

int dbd_st_rows {
  code
}



I'm having a frustrating day so far so perhaps have lost the ability to read 
diffs and C but in your change at
https://github.com/perl5-dbi/dbi/commit/29f6b9b76e9c637be31cb80f1a262ff68b42ef43#diff-cb6af96fe009d6f8d9d682415e1ab755

if retval0 (checked above)  I don't see where the checked above bit is.
it looks like:
if (retval == 0)
   ..
else if (retval == -1)
   ..
else if (retval = -2)
   ..
else
   new stuff here
   retval could still be negative just not -1 or -2


The else if (retval = -2) covers other negative values, doesn't it?


my mistake, as I said, I was not having a good day.


Also, maybe a little picky but the comment and DBIc_ROW_COUNT0 does not 
match the code.


Yeah, I was in two minds about that. I'll use DBIc_ROW_COUNT0 in
practice, but !=0 seemed a better fit for the experimental warning.


If no DBDs use DBIc_ROW_COUNT then that warning you've put in will do
nothing. I'd like to see a driver which does use DBIc_ROW_COUNT and if
there are none I'm happy to change DBD::ODBC initially to a) test the
diff you just applied and b) test the suggested fix.


That would be great. Thank you Martin!

Tim.



I'll happily make any changes you suggest and can test any changes you want to 
try out in DBI but I think there are still some issues to discuss 

Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-21 Thread Martin J. Evans

On 21/07/15 15:03, Tim Bunce wrote:

On Tue, Jul 21, 2015 at 01:33:34PM +0100, Martin J. Evans wrote:

Long, sorry.


No problem. The whole topic is a bit of a mess.


On 20/07/15 18:00, Tim Bunce wrote:

On Mon, Jul 20, 2015 at 02:54:53PM +0100, Martin J. Evans wrote:

On 20/07/15 14:15, Tim Bunce wrote:

I think that would work for me - I'm happy to test it our here if you want to 
give it a go.

IIRC, when this was last discussed the problem is that some drivers
might not set DBIc_ROW_COUNT so you can't just use DBIc_ROW_COUNT.


Hence the check that DBIc_ROW_COUNT is not zero. Since the DBI code sets
it to zero before the call, if it's non-zero after the call we can be
sure that the driver has set it.


In fact, I just checked, and DBD::ODBC does not seem to call
DBIc_ROW_COUNT other than to set it to 0 in ODBC.xsi (which is code

from DBI anyway). Does that sound right?

Nope. Is it setting the underlying structure member directly?


no. All it does is it has a RowCount member in its own imp_sth_st structure 
which is a SQLLEN (64 bits on 64 bit machines and 32 on 32). Then it:

o dbd_db_execute returns the number of rows or -1 or -2 (error)
   At the end of dbd_st_execute if the affected rows is bigger than INT_MAX and 
warnings are
   on, it warns the rowcount has been truncated and changes the row count to 
INT_MAX.


That's reasonable. Hopefully we can do better though.


o has odbc_st_rows (because it is defined in dbd_xsh.h and I believed
   you needed to implement most of these in the DBD) which casts the
   internal RowCount to an int as odbc_st_rows is defined as returning an int.


The DBI provides a default rows method, in C, that returns DBIc_ROW_COUNT.
So a driver that stores the row count in DBIc_ROW_COUNT doesn't need to
provide a rows method at all (if all it needs to do is return the count).

That translates into not defining the dbd_st_rows macro. If that's not
defined then the rows method in Driver.xst won't get compiled in so
there'll be no call to a driver-provided dbd_st_rows.


ok, so I'll try removing dbd_st_rows and whenever I call SQLRowCount I'll use 
the DBIc_ROW_COUNT macro.


DBD::ODBC also has its own odbc_rows which returns an IV to workaround this 
issue in DBI when I found it back in 2012.


If DBD::ODBC switched to using DBIc_ROW_COUNT then you could remove
dbd_st_rows/odbc_rows.  (It seems unlikely that sizeof(IV) would ever me
less than sizeof(SQLLEN) but that might be worth an assertion anyway.)


I will add assertion.




Looking at 'do' in DBI.pm it just does:

 sub do {
my($dbh, $statement, $attr, @params) = @_;
my $sth = $dbh-prepare($statement, $attr) or return undef;
$sth-execute(@params) or return undef;
my $rows = $sth-rows;
($rows == 0) ? 0E0 : $rows;
 }

so doesn't that just end up in dbd_st_rows?


Assuming the driver is using that default do() method, then it'll
end up in dbd_st_rows if the driver has defined a dbd_st_rows macro,
else it'll end up in the DBI's default rows() method.


If a driver is supposed to set DBIc_ROW_COUNT I'd rather change the
drivers I maintain to do that especially since in ODBC and 64bit
SQLRowCount already returns a 64 bit value.


Yeap. That's best.


See above, I don't see how that fits in right now.


Is the only outstanding issue now the 'int' return type on some various
dbd_st_* functions?


Yes, I believe so.


I tried to check my assumptions and this is what I found:

o DBD::ODBC has its own 'do' method because it can use SQLExecDirect instead of 
prepare/execute. This returns the rows affected correctly as it returns an SV 
created from the SQLLEN RowCount. So, the do method in DBI (shown above) is 
neither here nor there for DBD::ODBC.

o DBD::ODBC has a dbd_st_rows which seems to get called if someone calls the 
rows method.
dbd_st_rows is defined in dbd_xsh.h as returning an int so this is wrong.


And can simply be removed, per the above.


o 'execute' or dbd_st_execute returns the rows and is again defined in dbd_xsh 
as returning an int.

I don't see where DBIc_ROW_COUNT comes in unless you are saying every time a 
DBD discovers the row count it should call DBIc_ROW_COUNT macro.


DBIc_ROW_COUNT is just a macro for an IV in the imp_sth structure. Most,
if not all, compiled drivers that aren't using DBIc_ROW_COUNT are simply
using their own integer element in the imp_sth structure. In the case of
DBD::Pg that's declared as a plain int type.

So I'd hope and expect a driver can simply use DBIc_ROW_COUNT _instead of_
whatever it's currently using.


I also noticed something I should have seen before: dbd_st_rows() is
defined as returning an int. I _think_ it would be safe to change the
definition to returning an IV since it's only used internally by drivers
via the Driver.xst template file that does:

 XST_mIV(0, dbd_st_rows(sth, imp_sth));


Unless I'm missing something I think that will break most drivers as when I 
grepped cpan I found most drivers 

Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-20 Thread Martin J. Evans

On 19/07/15 15:41, Tim Bunce wrote:

On Thu, Jul 16, 2015 at 10:46:35AM -0700, David E. Wheeler wrote:

On Jul 16, 2015, at 6:40 AM, Tim Bunce tim.bu...@pobox.com wrote:


Well, this contains lots more light! ...


- dbd_st_execute for 03fdf4e0

parse_params statement

SELECT c.change_id ...



Binding parameters: SELECT c.change_id



-- do_error
Out of sort memory, consider increasing server sort buffer size error 1038 
recorded: Out of sort memory, consider increasing server sort buffer size
-- do_error
- dbd_st_execute returning imp_sth-row_num 18446744073709551615
!! ERROR: 1038 'Out of sort memory, consider increasing server sort buffer 
size' (err#0)
- execute= ( -1 ) [1 items] at /usr/lib/perl5/DBI.pm line 1632 via  at 
/usr/local/share/perl/5.18.2/App/Sqitch/Role/DBIEngine.pm line 149


So execute failed. Note the crazy row_num. Execute seems to have
returned -1, which is a true value.


!! The ERROR '1038' was CLEARED by call to fetchrow_hashref method
- fetchrow_hashref for DBD::mysql::st (DBI::st=HASH(0x42cfcc0)~0x4231cf8) 
thr#2603010


Then the higher-level code called fetchrow_hashref, which cleared the
error recorded by execute().


FWIW, the database handle is created like this:

 my $dbh = DBI-connect($uri-dbi_dsn, scalar $self-username, $pass, {
 PrintError   = 0,
 RaiseError   = 0,
 AutoCommit   = 1,
 mysql_enable_utf8= 1,
 mysql_auto_reconnect = 0,
 mysql_use_result = 0, # Prevent Commands out of sync error.
 HandleError  = sub {
 my ($err, $dbh) = @_;
 $@ = $err;
 @_ = ($dbh-state || 'DEV' = $dbh-errstr);
 goto hurl;
 },

Context: 
https://github.com/theory/sqitch/blob/master/lib/App/Sqitch/Engine/mysql.pm#L59

So I’m a little confused as to why the execute failure was ignored. Is this an 
issue with DBD::mysql?


Note the row_num 18446744073709551615 above, that's -1 as an unsigned 64 bit 
long.

DBD::mysql's handling of row_num seems less than ideal (prompted in part by
baggage of the DBI's ancient driver API).



int dbd_st_execute(SV* sth, imp_sth_t* imp_sth) == XXX int (forced by DBI API)
{
   ...
   imp_sth-row_num= mysql_st_internal_execute(...) == row_num is declared as 
my_ulonglong
   ...
   if (imp_sth-row_num+1 != (my_ulonglong)-1) { ... } == XXX
   ...
   ...
 sprintf(actual_row_num, %llu, imp_sth-row_num);
 PerlIO_printf(DBIc_LOGPIO(imp_xxh),  - dbd_st_execute returning 
imp_sth-row_num %s\n, actual_row_num);
   }
   return (int)imp_sth-row_num; # == XXX
}

my_ulonglong mysql_st_internal_execute(...) == unsigned
{
   my_ulonglong rows= 0; == unsigned

 if (!slen) {
   do_error(h, JW_ERR_QUERY, Missing table name ,NULL);
   return -2; == signed
 }
 if (!(table= malloc(slen+1))) {
   do_error(h, JW_ERR_MEM, Out of memory ,NULL);
   return -2; == signed
 }

 if (!(*result)) {
   do_error(h, mysql_errno(svsock), 
mysql_error(svsock),mysql_sqlstate(svsock));
   return -2; == signed
 }

   if(rows == -2) { == signed
 do_error(h, mysql_errno(svsock), mysql_error(svsock),
  mysql_sqlstate(svsock));
 if (DBIc_TRACE_LEVEL(imp_xxh) = 2)
   PerlIO_printf(DBIc_LOGPIO(imp_xxh), IGNORING ERROR errno %d\n, errno);
 rows = -2; == signed
   }
   return(rows);
}

mysql_st_internal_execute41(...) has very similar issues

Looks to me like you've hit some latent bugs in the DBD::mysql code (e.g., it's
not safe/reliable to throw negative numbers around in unsigned types) compounded
by the limitations of the ancient DBI driver API:
https://github.com/perl5-dbi/dbi/blob/1486773ec0bf357661d756cf37ff2988b5eaf24d/Driver.xst#L585-L601

Seems like there's a need to separate row count from execute return value.

Internally the DBI has a DBIc_ROW_COUNT(sth) macro that has an IV type.
That's a signed int that would be 64 bits on most modern systems.
On many of those systems the plain int type might be 32 bits.

I've just pushed an experimental change that might help in general
https://github.com/perl5-dbi/dbi/commit/29f6b9b76e9c637be31cb80f1a262ff68b42ef43#diff-cb6af96fe009d6f8d9d682415e1ab755
but probably wouldn't in your case.

At the moment I'd view this as a DBD::mysql bug.

Tim.

p.s. These open DBD::mysql issues might also be more or less related:
https://rt.cpan.org/Public/Bug/Display.html?id=48158
https://rt.cpan.org/Public/Bug/Display.html?id=80394
https://rt.cpan.org/Public/Bug/Display.html?id=75570




Please also see the issue I reported in DBI back in 2012:

https://rt.cpan.org/Ticket/Display.html?id=81911

I had to add various workarounds and a warning to DBD::ODBC.

Martin


Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-20 Thread Tim Bunce
On Mon, Jul 20, 2015 at 08:55:40AM +0100, Martin J. Evans wrote:
 On 19/07/15 15:41, Tim Bunce wrote:
 
 Please also see the issue I reported in DBI back in 2012:
 
 https://rt.cpan.org/Ticket/Display.html?id=81911
 
 I had to add various workarounds and a warning to DBD::ODBC.

Ah, thanks for the reminder Martin! I'll add a comment on that case.

Any thoughts about the general principle of changing the XS execute to
return the value of the DBIc_ROW_COUNT IV if the int returned by
dbd_st_execute is  0 and DBIc_ROW_COUNT  0?

Tim.


Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-20 Thread Tim Bunce
On Sun, Jul 19, 2015 at 06:39:59PM -0700, David E. Wheeler wrote:
 On Jul 19, 2015, at 7:41 AM, Tim Bunce tim.bu...@pobox.com wrote:
 
  Internally the DBI has a DBIc_ROW_COUNT(sth) macro that has an IV type.
  That's a signed int that would be 64 bits on most modern systems.
  On many of those systems the plain int type might be 32 bits.
  
  I've just pushed an experimental change that might help in general
  https://github.com/perl5-dbi/dbi/commit/29f6b9b76e9c637be31cb80f1a262ff68b42ef43#diff-cb6af96fe009d6f8d9d682415e1ab755
  but probably wouldn't in your case.
 
 Huh. Why not?

That change just adds a warning.

  At the moment I'd view this as a DBD::mysql bug.

The assignment of signed values to unsigned types in DBD::mysql ought to
be fixed.

  p.s. These open DBD::mysql issues might also be more or less related:
  https://rt.cpan.org/Public/Bug/Display.html?id=48158
  https://rt.cpan.org/Public/Bug/Display.html?id=80394
  https://rt.cpan.org/Public/Bug/Display.html?id=75570
 
 Given that these have had exactly 0 activity in three years, how should we go 
 about getting thins on the maintaners’ radar?

I don't know. The repo is managed by the perl5-dbi org on github.
https://github.com/perl5-dbi/DBD-mysql so in theory any of the team
members https://github.com/orgs/perl5-dbi/people could commit patches.
Patrick Galbraith, the primary maintainer (CC'd) seems fairly active at
the moment so I'd start by asking Patrick for his thoughts.

 Also, is there something I can do in Sqitch to work around this issue?

I'm not sure. It's possible that a HandleSetError handler could help
https://metacpan.org/pod/DBI#HandleSetErr

(It's also possible that I'm misreading the cause as I'm not clear how
the -2 becomes a -1 but I gave up looking further after seeing the
problems with the current DBD::mysql code.)

Tim.


Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-20 Thread Martin J. Evans

On 20/07/15 14:15, Tim Bunce wrote:

On Mon, Jul 20, 2015 at 08:55:40AM +0100, Martin J. Evans wrote:

On 19/07/15 15:41, Tim Bunce wrote:

Please also see the issue I reported in DBI back in 2012:

https://rt.cpan.org/Ticket/Display.html?id=81911

I had to add various workarounds and a warning to DBD::ODBC.


Ah, thanks for the reminder Martin! I'll add a comment on that case.

Any thoughts about the general principle of changing the XS execute to
return the value of the DBIc_ROW_COUNT IV if the int returned by
dbd_st_execute is  0 and DBIc_ROW_COUNT  0?

Tim.



I think that would work for me - I'm happy to test it our here if you want to 
give it a go.

IIRC, when this was last discussed the problem is that some drivers might not 
set DBIc_ROW_COUNT so you can't just use DBIc_ROW_COUNT. In fact, I just 
checked, and DBD::ODBC does not seem to call DBIc_ROW_COUNT other than to set 
it to 0 in ODBC.xsi (which is code from DBI anyway). Does that sound right?

If a driver is supposed to set DBIc_ROW_COUNT I'd rather change the drivers I 
maintain to do that especially since in ODBC and 64bit SQLRowCount already 
returns a 64 bit value. Is there some docs on that or perhaps you could just 
tell me or point me at a driver that does it correctly.

I'm having a frustrating day so far so perhaps have lost the ability to read 
diffs and C but in your change at
https://github.com/perl5-dbi/dbi/commit/29f6b9b76e9c637be31cb80f1a262ff68b42ef43#diff-cb6af96fe009d6f8d9d682415e1ab755

if retval0 (checked above)  I don't see where the checked above bit is.

it looks like:

if (retval == 0)
  ..
else if (retval == -1)
  ..
else if (retval = -2)
  ..
else
  new stuff here
  retval could still be negative just not -1 or -2

Also, maybe a little picky but the comment and DBIc_ROW_COUNT0 does not 
match the code.

If no DBDs use DBIc_ROW_COUNT then that warning you've put in will do nothing. 
I'd like to see a driver which does use DBIc_ROW_COUNT and if there are none 
I'm happy to change DBD::ODBC initially to a) test the diff you just applied 
and b) test the suggested fix.

Martin


Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-20 Thread Tim Bunce
On Mon, Jul 20, 2015 at 02:54:53PM +0100, Martin J. Evans wrote:
 On 20/07/15 14:15, Tim Bunce wrote:
 
 I think that would work for me - I'm happy to test it our here if you want to 
 give it a go.
 
 IIRC, when this was last discussed the problem is that some drivers
 might not set DBIc_ROW_COUNT so you can't just use DBIc_ROW_COUNT.

Hence the check that DBIc_ROW_COUNT is not zero. Since the DBI code sets
it to zero before the call, if it's non-zero after the call we can be
sure that the driver has set it.

 In fact, I just checked, and DBD::ODBC does not seem to call
 DBIc_ROW_COUNT other than to set it to 0 in ODBC.xsi (which is code
 from DBI anyway). Does that sound right?

Nope. Is it setting the underlying structure member directly?

 If a driver is supposed to set DBIc_ROW_COUNT I'd rather change the
 drivers I maintain to do that especially since in ODBC and 64bit
 SQLRowCount already returns a 64 bit value.

Yeap. That's best.

 Is there some docs on that or perhaps you could just tell me or point
 me at a driver that does it correctly.

No docs, sadly. And I'm not aware of any drivers that do.

I took a look at DBD:Pg and that uses it's own 'rows' structure
member which is defined as an int, and int is used in the code.

I also noticed something I should have seen before: dbd_st_rows() is
defined as returning an int. I _think_ it would be safe to change the
definition to returning an IV since it's only used internally by drivers
via the Driver.xst template file that does:

XST_mIV(0, dbd_st_rows(sth, imp_sth));

 I'm having a frustrating day so far so perhaps have lost the ability to read 
 diffs and C but in your change at
 https://github.com/perl5-dbi/dbi/commit/29f6b9b76e9c637be31cb80f1a262ff68b42ef43#diff-cb6af96fe009d6f8d9d682415e1ab755
 
 if retval0 (checked above)  I don't see where the checked above bit is.
 it looks like:
 if (retval == 0)
   ..
 else if (retval == -1)
   ..
 else if (retval = -2)
   ..
 else
   new stuff here
   retval could still be negative just not -1 or -2

The else if (retval = -2) covers other negative values, doesn't it?

 Also, maybe a little picky but the comment and DBIc_ROW_COUNT0 does not 
 match the code.

Yeah, I was in two minds about that. I'll use DBIc_ROW_COUNT0 in
practice, but !=0 seemed a better fit for the experimental warning.

 If no DBDs use DBIc_ROW_COUNT then that warning you've put in will do
 nothing. I'd like to see a driver which does use DBIc_ROW_COUNT and if
 there are none I'm happy to change DBD::ODBC initially to a) test the
 diff you just applied and b) test the suggested fix.

That would be great. Thank you Martin!

Tim.


Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-20 Thread Tim Bunce
On Mon, Jul 20, 2015 at 06:00:53PM +0100, Tim Bunce wrote:
 On Mon, Jul 20, 2015 at 02:54:53PM +0100, Martin J. Evans wrote:
 
 I also noticed something I should have seen before: dbd_st_rows() is
 defined as returning an int. I _think_ it would be safe to change the
 definition to returning an IV since it's only used internally by drivers
 via the Driver.xst template file that does:
 
 XST_mIV(0, dbd_st_rows(sth, imp_sth));

Ah. The same logic also means I could change the return type of
dbd_st_execute and dbd_db_do4, which also would help.

Tim.


Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-19 Thread David E . Wheeler
On Jul 19, 2015, at 7:41 AM, Tim Bunce tim.bu...@pobox.com wrote:

 Internally the DBI has a DBIc_ROW_COUNT(sth) macro that has an IV type.
 That's a signed int that would be 64 bits on most modern systems.
 On many of those systems the plain int type might be 32 bits.
 
 I've just pushed an experimental change that might help in general
 https://github.com/perl5-dbi/dbi/commit/29f6b9b76e9c637be31cb80f1a262ff68b42ef43#diff-cb6af96fe009d6f8d9d682415e1ab755
 but probably wouldn't in your case.

Huh. Why not?

 At the moment I'd view this as a DBD::mysql bug.
 
 Tim.
 
 p.s. These open DBD::mysql issues might also be more or less related:
 https://rt.cpan.org/Public/Bug/Display.html?id=48158
 https://rt.cpan.org/Public/Bug/Display.html?id=80394
 https://rt.cpan.org/Public/Bug/Display.html?id=75570

Given that these have had exactly 0 activity in three years, how should we go 
about getting thins on the maintaners’ radar?

Also, is there something I can do in Sqitch to work around this issue?

Thanks,

David

DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-19 Thread Tim Bunce
On Thu, Jul 16, 2015 at 10:46:35AM -0700, David E. Wheeler wrote:
 On Jul 16, 2015, at 6:40 AM, Tim Bunce tim.bu...@pobox.com wrote:
 
  Well, this contains lots more light! ...
  
  - dbd_st_execute for 03fdf4e0
  parse_params statement 
 SELECT c.change_id ...
  
  Binding parameters: SELECT c.change_id
  
 -- do_error
  Out of sort memory, consider increasing server sort buffer size error 
  1038 recorded: Out of sort memory, consider increasing server sort buffer 
  size
 -- do_error
  - dbd_st_execute returning imp_sth-row_num 18446744073709551615
 !! ERROR: 1038 'Out of sort memory, consider increasing server sort 
  buffer size' (err#0)
 - execute= ( -1 ) [1 items] at /usr/lib/perl5/DBI.pm line 1632 via  
  at /usr/local/share/perl/5.18.2/App/Sqitch/Role/DBIEngine.pm line 149
  
  So execute failed. Note the crazy row_num. Execute seems to have
  returned -1, which is a true value.
  
 !! The ERROR '1038' was CLEARED by call to fetchrow_hashref method
 - fetchrow_hashref for DBD::mysql::st 
  (DBI::st=HASH(0x42cfcc0)~0x4231cf8) thr#2603010
  
  Then the higher-level code called fetchrow_hashref, which cleared the
  error recorded by execute().
 
 FWIW, the database handle is created like this:
 
 my $dbh = DBI-connect($uri-dbi_dsn, scalar $self-username, $pass, {
 PrintError   = 0,
 RaiseError   = 0,
 AutoCommit   = 1,
 mysql_enable_utf8= 1,
 mysql_auto_reconnect = 0,
 mysql_use_result = 0, # Prevent Commands out of sync error.
 HandleError  = sub {
 my ($err, $dbh) = @_;
 $@ = $err;
 @_ = ($dbh-state || 'DEV' = $dbh-errstr);
 goto hurl;
 },
 
 Context: 
 https://github.com/theory/sqitch/blob/master/lib/App/Sqitch/Engine/mysql.pm#L59
 
 So I’m a little confused as to why the execute failure was ignored. Is this 
 an issue with DBD::mysql?

Note the row_num 18446744073709551615 above, that's -1 as an unsigned 64 bit 
long.

DBD::mysql's handling of row_num seems less than ideal (prompted in part by
baggage of the DBI's ancient driver API).

int dbd_st_execute(SV* sth, imp_sth_t* imp_sth) == XXX int (forced by DBI API)
{
  ...
  imp_sth-row_num= mysql_st_internal_execute(...) == row_num is declared as 
my_ulonglong
  ...
  if (imp_sth-row_num+1 != (my_ulonglong)-1) { ... } == XXX
  ...
  ...
sprintf(actual_row_num, %llu, imp_sth-row_num);
PerlIO_printf(DBIc_LOGPIO(imp_xxh),  - dbd_st_execute returning 
imp_sth-row_num %s\n, actual_row_num);
  }
  return (int)imp_sth-row_num; # == XXX
}

my_ulonglong mysql_st_internal_execute(...) == unsigned
{
  my_ulonglong rows= 0; == unsigned
 
if (!slen) {
  do_error(h, JW_ERR_QUERY, Missing table name ,NULL);
  return -2; == signed
}
if (!(table= malloc(slen+1))) {
  do_error(h, JW_ERR_MEM, Out of memory ,NULL);
  return -2; == signed
}
 
if (!(*result)) {
  do_error(h, mysql_errno(svsock), 
mysql_error(svsock),mysql_sqlstate(svsock));
  return -2; == signed
}
 
  if(rows == -2) { == signed
do_error(h, mysql_errno(svsock), mysql_error(svsock), 
 mysql_sqlstate(svsock));
if (DBIc_TRACE_LEVEL(imp_xxh) = 2)
  PerlIO_printf(DBIc_LOGPIO(imp_xxh), IGNORING ERROR errno %d\n, errno);
rows = -2; == signed
  }
  return(rows);
}

mysql_st_internal_execute41(...) has very similar issues

Looks to me like you've hit some latent bugs in the DBD::mysql code (e.g., it's
not safe/reliable to throw negative numbers around in unsigned types) compounded
by the limitations of the ancient DBI driver API:
https://github.com/perl5-dbi/dbi/blob/1486773ec0bf357661d756cf37ff2988b5eaf24d/Driver.xst#L585-L601

Seems like there's a need to separate row count from execute return value.

Internally the DBI has a DBIc_ROW_COUNT(sth) macro that has an IV type.
That's a signed int that would be 64 bits on most modern systems.
On many of those systems the plain int type might be 32 bits.

I've just pushed an experimental change that might help in general
https://github.com/perl5-dbi/dbi/commit/29f6b9b76e9c637be31cb80f1a262ff68b42ef43#diff-cb6af96fe009d6f8d9d682415e1ab755
but probably wouldn't in your case.

At the moment I'd view this as a DBD::mysql bug.

Tim.

p.s. These open DBD::mysql issues might also be more or less related:
https://rt.cpan.org/Public/Bug/Display.html?id=48158
https://rt.cpan.org/Public/Bug/Display.html?id=80394
https://rt.cpan.org/Public/Bug/Display.html?id=75570


Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-16 Thread David E. Wheeler
On Jul 16, 2015, at 6:40 AM, Tim Bunce tim.bu...@pobox.com wrote:

 Well, this contains lots more light! ...
 
 - dbd_st_execute for 03fdf4e0
 parse_params statement 
SELECT c.change_id ...
 
 Binding parameters: SELECT c.change_id
 
-- do_error
 Out of sort memory, consider increasing server sort buffer size error 1038 
 recorded: Out of sort memory, consider increasing server sort buffer size
-- do_error
 - dbd_st_execute returning imp_sth-row_num 18446744073709551615
!! ERROR: 1038 'Out of sort memory, consider increasing server sort 
 buffer size' (err#0)
- execute= ( -1 ) [1 items] at /usr/lib/perl5/DBI.pm line 1632 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Role/DBIEngine.pm line 149
 
 So execute failed. Note the crazy row_num. Execute seems to have
 returned -1, which is a true value.
 
!! The ERROR '1038' was CLEARED by call to fetchrow_hashref method
- fetchrow_hashref for DBD::mysql::st 
 (DBI::st=HASH(0x42cfcc0)~0x4231cf8) thr#2603010
 
 Then the higher-level code called fetchrow_hashref, which cleared the
 error recorded by execute().

FWIW, the database handle is created like this:

my $dbh = DBI-connect($uri-dbi_dsn, scalar $self-username, $pass, {
PrintError   = 0,
RaiseError   = 0,
AutoCommit   = 1,
mysql_enable_utf8= 1,
mysql_auto_reconnect = 0,
mysql_use_result = 0, # Prevent Commands out of sync error.
HandleError  = sub {
my ($err, $dbh) = @_;
$@ = $err;
@_ = ($dbh-state || 'DEV' = $dbh-errstr);
goto hurl;
},

Context: 
https://github.com/theory/sqitch/blob/master/lib/App/Sqitch/Engine/mysql.pm#L59

So I’m a little confused as to why the execute failure was ignored. Is this an 
issue with DBD::mysql?

I assume the OP’s server could use some tuning. Seems pretty weird, though.

Thanks,

David

smime.p7s
Description: S/MIME cryptographic signature


Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-16 Thread Martin J. Evans

On 15/07/15 18:49, David E. Wheeler wrote:

On Jul 14, 2015, at 3:24 AM, Tim Bunce tim.bu...@pobox.com wrote:


I can't see anything obvious from this trace. Come back with a level 4
trace and hopefully that'll shed sufficient light.


Here we go.



I presume you saw the

Out of sort memory, consider increasing server sort buffer size error 1038 
recorded: Out of sort memory, consider increasing server sort buffer size

Martin


Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-16 Thread Tim Bunce
On Wed, Jul 15, 2015 at 10:49:45AM -0700, David E. Wheeler wrote:
 On Jul 14, 2015, at 3:24 AM, Tim Bunce tim.bu...@pobox.com wrote:
 
  I can't see anything obvious from this trace. Come back with a level 4
  trace and hopefully that'll shed sufficient light.
 
 Here we go.
 
  $ DBI_TRACE=4 sqitch status

Well, this contains lots more light! ...

   - dbd_st_execute for 03fdf4e0
  parse_params statement 
  SELECT c.change_id ...

  Binding parameters: SELECT c.change_id

  -- do_error
  Out of sort memory, consider increasing server sort buffer size error 1038 
  recorded: Out of sort memory, consider increasing server sort buffer size
  -- do_error
   - dbd_st_execute returning imp_sth-row_num 18446744073709551615
  !! ERROR: 1038 'Out of sort memory, consider increasing server sort 
  buffer size' (err#0)
  - execute= ( -1 ) [1 items] at /usr/lib/perl5/DBI.pm line 1632 via  at 
  /usr/local/share/perl/5.18.2/App/Sqitch/Role/DBIEngine.pm line 149

So execute failed. Note the crazy row_num. Execute seems to have
returned -1, which is a true value.

  !! The ERROR '1038' was CLEARED by call to fetchrow_hashref method
  - fetchrow_hashref for DBD::mysql::st 
  (DBI::st=HASH(0x42cfcc0)~0x4231cf8) thr#2603010

Then the higher-level code called fetchrow_hashref, which cleared the
error recorded by execute().

Then it all goes downhill from there.

Tim.

  1   - mysql_async_ready for DBD::mysql::st (DBI::st=HASH(0x4231cf8)~INNER) 
  thr#2603010
  -- do_error
  Handle is not in asynchronous mode error 2000 recorded: Handle is not in 
  asynchronous mode
  -- do_error
  !! ERROR: 2000 'Handle is not in asynchronous mode' (err#0)
  1   - mysql_async_ready= ( undef ) [1 items] at 
  /usr/local/lib/perl/5.18.2/DBD/mysql.pm line 864 via  at 
  /usr/local/share/perl/5.18.2/App/Sqitch/Role/DBIEngine.pm line 116
  1   - FETCH for DBD::mysql::st (DBI::st=HASH(0x4231cf8)~INNER 'NAME') 
  thr#2603010
  - dbd_st_FETCH_attrib for 03fdf4e0, key NAME
  -- do_error
  statement contains no result error 4 recorded: statement contains no result
  -- do_error
  !! ERROR: 4 'statement contains no result' (err#0)
  1   - FETCH= ( undef ) [1 items] at 
  /usr/local/lib/perl/5.18.2/DBD/mysql.pm line 867 via  at 
  /usr/local/share/perl/5.18.2/App/Sqitch/Role/DBIEngine.pm line 116
  !! The ERROR '4' was CLEARED by call to fetch method
  1   - fetch for DBD::mysql::st (DBI::st=HASH(0x4231cf8)~INNER) thr#2603010
  - dbd_st_fetch
  dbd_st_fetch for 04243568, chopblanks 0
  -- do_error
  fetch() without execute() error 19 recorded: fetch() without execute()
  -- do_error
  !! ERROR: 19 'fetch() without execute()' (err#0)


Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-15 Thread David E. Wheeler
On Jul 14, 2015, at 3:24 AM, Tim Bunce tim.bu...@pobox.com wrote:

 I can't see anything obvious from this trace. Come back with a level 4
 trace and hopefully that'll shed sufficient light.

Here we go.

 $ DBI_TRACE=4 sqitch status
 DBI 1.630-ithread default trace level set to 0x0/4 (pid 10670 pi 2603010) 
 at DBI.pm line 288 via DBIEngine.pm line 7
 # On database sqitch_testing
 - DBI-connect(dbi:mysql:database=sqitch, root, , HASH(0x4231ae8))
 - DBI-install_driver(mysql) for linux perl=5.018002 pid=10670 ruid=1000 
 euid=1000
install_driver: DBD::mysql version 4.031 loaded from 
 /usr/local/lib/perl/5.18.2/DBD/mysql.pm
 - install_driver= DBI::dr=HASH(0x3341728)
 !! The warn '0' was CLEARED by call to connect method
 - connect for DBD::mysql::dr (DBI::dr=HASH(0x3341728)~0x33417e8 
 'database=sqitch' 'root'  HASH(0x3f68f70)) thr#2603010
 imp_dbh-connect: dsn = database=sqitch, uid = root, pwd = root
 imp_dbh-my_login : dbname = sqitch, uid = root, pwd = root,host = NULL, port 
 = NULL
 imp_dbh-mysql_dr_connect: host = |NULL|, port = 0, uid = root, pwd = root
 imp_dbh-use_mysql_use_result: 0
 imp_dbh-bind_type_guessing: 0
 imp_dbh-use_server_side_prepare: 0
 mysql_options: MYSQL_SET_CHARSET_NAME=utf8
 imp_dbh-mysql_dr_connect: client_flags = 2
 imp_dbh-mysql_dr_connect: -- connect= ( DBI::db=HASH(0x42cfc78) ) [1 
 items] at /usr/lib/perl5/DBI.pm line 671 via  at (eval 231) line 58
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'RaiseError' 
 0) thr#2603010
 STORE DBI::db=HASH(0x42cfc00) 'RaiseError' = 0
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 723 via  at (eval 
 231) line 58
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'PrintError' 
 0) thr#2603010
 STORE DBI::db=HASH(0x42cfc00) 'PrintError' = 0
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 723 via  at (eval 
 231) line 58
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'AutoCommit' 
 1) thr#2603010
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 723 via  at (eval 
 231) line 58
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 
 'mysql_auto_reconnect' 0) thr#2603010
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'Username' 
 'root') thr#2603010
 STORE DBI::db=HASH(0x42cfc00) 'Username' = 'root'
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'HandleError' 
 CODE(0x3ebbb10)) thr#2603010
 STORE DBI::db=HASH(0x42cfc00) 'HandleError' = CODE(0x3ebbb10)
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 
 'mysql_enable_utf8' 1) thr#2603010
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'Callbacks' 
 HASH(0x421a780)) thr#2603010
 STORE DBI::db=HASH(0x42cfc00) 'Callbacks' = HASH(0x421a780)
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
 - STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 
 'mysql_use_result' 0) thr#2603010
 - STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
 {{ connected callback CODE(0x3ebbd08) being invoked
 1   - do for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'SET SESSION 
 character_set_client   = 'utf8'') thr#2603010
 mysql.xs do() use_server_side_prepare 0, async 0
 mysql_st_internal_execute MYSQL_VERSION_ID 50543
 parse_params statement SET SESSION character_set_client   = 'utf8'
 1   - do= ( '0E0' ) [1 items] at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 75 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
 1   - do for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'SET SESSION 
 character_set_server   = 'utf8'') thr#2603010
 mysql.xs do() use_server_side_prepare 0, async 0
 mysql_st_internal_execute MYSQL_VERSION_ID 50543
 parse_params statement SET SESSION character_set_server   = 'utf8'
 1   - do= ( '0E0' ) [1 items] at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 75 via  at 
 /usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
 1   - do for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'SET SESSION 
 time_zone  = '+00:00'') thr#2603010
 mysql.xs do() use_server_side_prepare 0, async 0
 mysql_st_internal_execute MYSQL_VERSION_ID 50543
 parse_params statement SET SESSION time_zone  

Re: Why is selectrow_hashref complaining about a fetch without execute?

2015-07-14 Thread Tim Bunce
On Mon, Jul 13, 2015 at 03:53:43PM -0700, David E. Wheeler wrote:
DBIers,
Got a complaint about a “fetch() without execute()” error. Asked for a 
 trace, got this. Looks like
it’s coming from selectrow_hashref()? That shouldn’t happen, right?
 
  $ DBI_TRACE=1 sqitch status

Trace level 1 is handy when you just want a summary of what's going on.
If something odd is happening then you'd want at least trace level 2
so you see the method _calls_ as well as the _returns_.
For investigating mysterious happenings then higher trace levels,
like 4 thru 15, are more appropriate. 4 is usually sufficient.

I can't see anything obvious from this trace. Come back with a level 4
trace and hopefully that'll shed sufficient light.

Tim.

  - STORE('dbi_connect_closure', CODE(0x1c11b10))= ( 1 ) [1 items] at 
 DBI.pm line 742
  - FETCH('mysql_serverinfo')= ( '5.6.17-0ubuntu0.14.04.1-log' ) [1 
 items] at mysql.pm line 106
  - FETCH('mysql_serverversion')= ( '50617' ) [1 items] at mysql.pm line 
 109
  !! The ERROR '1038' was CLEARED by call to fetchrow_hashref method
  !! The ERROR '4' was CLEARED by call to fetch method
 ERROR: 19 'fetch() without execute()' (err#0)
  - DESTROY(DBI::st=HASH(0x2ae9638))= ( undef ) [1 items] at DBI.pm line 
 1639
  !! ERROR: 19 'fetch() without execute()' (err#0)
  - selectrow_hashref('
  SELECT c.change_id ...
   , c.planned_a...', undef, ...)= ( undef ) [1 items] at 
 DBIEngine.pm line 116
  - HandleError on DBI::db=HASH(0x2b9ea48) via CODE(0x278aa80) (undef)
  - $DBI::state= 'S1000'
  - $DBI::state= 'S1000'
  - $DBI::state= 'S1000'
  - $DBI::state= 'S1000'
  - $DBI::state= 'S1000'
  !! The ERROR '19' was CLEARED by call to selectcol_arrayref method
  - DESTROY(DBI::st=HASH(0x2ae94b8))= ( undef ) [1 items] at mysql.pm 
 line 218
  - selectcol_arrayref('
  SELECT COUNT(*)
FROM information_schema.tables
   WHERE table_schema = ?
 AND table_name   = ?
  ', undef, ...)= ( [ '1' ] ) [1 items] at mysql.pm line 218
  - DESTROY(DBI::db=HASH(0x2b9ea48))= ( undef ) [1 items] at Tiny.pm line 
 88
  fetch() without execute()
  - disconnect_all= ( ) [0 items] (not implemented) at DBI.pm line 750
  !   - DESTROY(DBI::dr=HASH(0x1c119a8))= ( undef ) [1 items] during global 
 destruction
 
Thanks,
David
Reverse link: [1]unknown
 
 References
 
Visible links
1. https://github.com/theory/sqitch/issues/235#issuecomment-120180277




Why is selectrow_hashref complaining about a fetch without execute?

2015-07-13 Thread David E. Wheeler
DBIers,

Got a complaint about a “fetch() without execute()” error. Asked for a trace, 
got this. Looks like it’s coming from selectrow_hashref()? That shouldn’t 
happen, right?

 $ DBI_TRACE=1 sqitch status
 DBI 1.630-ithread default trace level set to 0x0/1 (pid 3381 pi ed2010) 
 at DBI.pm line 288 via DBIEngine.pm line 7
 # On database sqitch_testing
 - DBI-connect(dbi:mysql:database=sqitch, root, , HASH(0x2b00a28))
 - DBI-install_driver(mysql) for linux perl=5.018002 pid=3381 ruid=1000 
 euid=1000
install_driver: DBD::mysql version 4.031 loaded from 
 /usr/local/lib/perl/5.18.2/DBD/mysql.pm
 - install_driver= DBI::dr=HASH(0x1c118e8)
 !! The warn '0' was CLEARED by call to connect method
 - connect('database=sqitch', 'root', ...)= ( DBI::db=HASH(0x2b9eac0) ) 
 [1 items] at DBI.pm line 671
 - STORE('RaiseError', 0)= ( 1 ) [1 items] at DBI.pm line 723
 - STORE('PrintError', 0)= ( 1 ) [1 items] at DBI.pm line 723
 - STORE('AutoCommit', 1)= ( 1 ) [1 items] at DBI.pm line 723
 - STORE('HandleError', CODE(0x278aa80))= ( 1 ) [1 items] at DBI.pm line 
 726
 - STORE('mysql_auto_reconnect', 0)= ( 1 ) [1 items] at DBI.pm line 726
 - STORE('Callbacks', HASH(0x2ae94d0))= ( 1 ) [1 items] at DBI.pm line 726
 - STORE('mysql_enable_utf8', 1)= ( 1 ) [1 items] at DBI.pm line 726
 - STORE('mysql_use_result', 0)= ( 1 ) [1 items] at DBI.pm line 726
 - STORE('Username', 'root')= ( 1 ) [1 items] at DBI.pm line 726
 {{ connected callback CODE(0x278ac78) being invoked
 }} connected callback CODE(0x278ac78) returned
 - connected('dbi:mysql:database=sqitch', 'root', ...)= ( undef ) [1 
 items] at DBI.pm line 733
 - connect= DBI::db=HASH(0x2b9eac0)
 - STORE('dbi_connect_closure', CODE(0x1c11b10))= ( 1 ) [1 items] at 
 DBI.pm line 742
 - FETCH('mysql_serverinfo')= ( '5.6.17-0ubuntu0.14.04.1-log' ) [1 items] 
 at mysql.pm line 106
 - FETCH('mysql_serverversion')= ( '50617' ) [1 items] at mysql.pm line 
 109
 !! The ERROR '1038' was CLEARED by call to fetchrow_hashref method
 !! The ERROR '4' was CLEARED by call to fetch method
ERROR: 19 'fetch() without execute()' (err#0)
 - DESTROY(DBI::st=HASH(0x2ae9638))= ( undef ) [1 items] at DBI.pm line 
 1639
 !! ERROR: 19 'fetch() without execute()' (err#0)
 - selectrow_hashref('
 SELECT c.change_id
  , c.script_hash
  , c.change
  , c.project
  , c.note
  , c.committer_name
  , c.committer_email
  , date_format(c.committed_at, 
 'year:%Y:month:%m:day:%d:hour:%H:minute:%i:second:%S:time_zone:UTC') AS 
 committed_at
  , c.planner_name
  , c.planner_email
  , date_format(c.planned_at, 
 'year:%Y:month:%m:day:%d:hour:%H:minute:%i:second:%S:time_zone:UTC') AS 
 planned_at
  , group_concat(t.tag SEPARATOR ' ') AS tags
   FROM changes   c
   LEFT JOIN tags t ON c.change_id = t.change_id
  WHERE c.project = ?
  GROUP BY c.change_id
  , c.script_hash
  , c.change
  , c.project
  , c.note
  , c.committer_name
  , c.committer_email
  , c.committed_at
  , c.planner_name
  , c.planner_email
  , c.planned_a...', undef, ...)= ( undef ) [1 items] at 
 DBIEngine.pm line 116
 - HandleError on DBI::db=HASH(0x2b9ea48) via CODE(0x278aa80) (undef)
 - $DBI::state= 'S1000'
 - $DBI::state= 'S1000'
 - $DBI::state= 'S1000'
 - $DBI::state= 'S1000'
 - $DBI::state= 'S1000'
 !! The ERROR '19' was CLEARED by call to selectcol_arrayref method
 - DESTROY(DBI::st=HASH(0x2ae94b8))= ( undef ) [1 items] at mysql.pm line 
 218
 - selectcol_arrayref('
 SELECT COUNT(*)
   FROM information_schema.tables
  WHERE table_schema = ?
AND table_name   = ?
 ', undef, ...)= ( [ '1' ] ) [1 items] at mysql.pm line 218
 - DESTROY(DBI::db=HASH(0x2b9ea48))= ( undef ) [1 items] at Tiny.pm line 
 88
 fetch() without execute()
 - disconnect_all= ( ) [0 items] (not implemented) at DBI.pm line 750
 !   - DESTROY(DBI::dr=HASH(0x1c119a8))= ( undef ) [1 items] during global 
 destruction
Thanks,

David



smime.p7s
Description: S/MIME cryptographic signature