Re: $dbh-{CachedKids}
On May 2, 2008, at 15:24, Tim Bunce wrote: You've not given me much to go on, but I'd guess it's related to the timing of when perl invokes the DESTROY method (which has changed between perl versions). In which case it may be mostly beyond the control of the DBI. A small self-contained example that behaves differently between different DBI/Perl versions will buy you more of my time :) Sorry Tim, I was hoping that that would be enough to shake some neurons loose for an easy answer. Here is my original query about this issue: http://aspn.activestate.com/ASPN/Mail/Message/perl-DBI-Dev/1359462 And here is your relevant reply: http://aspn.activestate.com/ASPN/Mail/Message/perl-DBI-dev/1449508 Only now I'm seeing that test fail on 5.6.2. Is it possible that 5.6.2 destroys the handle by the type the do() is called? I don't see your comment in 07kids.t anymore. Thanks, David
Re: $dbh-{CachedKids}
On May 9, 2008, at 04:42, Tim Bunce wrote: Only now I'm seeing that test fail on 5.6.2. Is it possible that 5.6.2 destroys the handle by the type the do() is called? Seems likely. Yes, I've released a new version where the test checks for $] 5.6.1 instead of 5.8.0. Got removed in May 2004 http://svn.perl.org/viewvc/modules/dbi/trunk/t/07kids.t?r1=333r2=335 Hrm. I guess it wasn't telling us much, since it's dependent on Perl versions, rather than the implementation of the DBI. If you really need to capture the complete state at the time the error *is recorded* rather than the time its *checked for and thrown* then you could use HandleSetErr instead of HandleError: http://search.cpan.org/~timb/DBI-1.604/DBI.pm#HandleSetErr_(code_ref,_inherited) Interesting, I hadn't seen that. As for my uses, I just wanted to get my tests passing again. No one has complained about the state of the error information in Exception::Class::DBI, to my knowledge. So I think I'll leave it as-is. Best, David
Function Calling Methods
Howdy dbi-devers,
More and more lately, I'm writing database functions in PL/pgSQL (in
PostgreSQL) or SQL (in MySQL and others) to do the heavy lifting of
interacting with database tables. I've been thinking that I'd really
love a DBI method to call these functions without having to do the
usual prepare / execute / fetch drill. Even using do() or
fetchrow_array() seems a bit silly in this context:
my ($source_id) = $dbh-fetchrow_array(
'SELECT get_source_id(?)',
undef,
$source,
);
What I'd love is a couple of DBI methods to do this for me. I
recognize that this is currently not defined by the DBI, but I'm
wondering whether it might be time. I've no idea whether JDBC
implements such an interface, but I was thinking of something like
this for function calls:
sub call {
my $dbh = shift;
my $func = shift;
my $places = join ', ', ('?') x @_;
return $dbh-fetchrow_array(
SELECT $func( $places ),
undef,
@_
);
}
This would allow me to call a function like so:
my $val = $dbh-call('get_source_id', $source );
Which is a much nicer syntax. Drivers might have to modify it, of
course; for MySQL, it should use CALL rather than SELECT.
For functions or procedures that happen to return sets or a cursor,
perhaps we could have a separate method that just returns a statement
handle that's ready to be fetched from?
sub cursor {
my $dbh = shift;
my $func = shift;
my $places = join ', ', ('?') x @_;
my $sth = $dbh-prepare( SELECT $func( $places ) );
$sth-execute(@_);
return $sth;
}
Just some ideas. I'm sure that there are more complications than this,
but even if we could just have something that handles simple functions
(think last_insert_id() -- eliminate this special case!), I think it'd
go a long way toward not only simplifying the use of database
functions in the DBI, but also toward encouraging DBI users to
actually make more use of database functions.
Thoughts?
Thanks,
David
Re: Function Calling Methods
On Wed, 14 May 2008 10:05:22 -0700, Martin Evens wrote:
That is slightly more complicated than it looks. DBD::Oracle already
magics a sth into existence for reference cursors but some databases
can return more than one result-set from a procedure - e.g., SQL
Server and the SQLMoreResults call to move to the next one.
Huh. How interesting.
I'm sure it's more complicated than it looks, to be sure, but I was
just trying to provide an 80% solution for simple functions that
return a scalar or a result set.
I have hundreds of functions and procedures in various packages in
Oracle we use via DBD::Oracle. We have no SQL at all outside
database functions/procedures/packages i.e., our Perl does not know
anything at all about the tables or columns in the database and the
only SQL executed is to prepare/execute procedures and functions.
Yes, this is exactly the sort of code I'm starting to write.
We wrap calls to functions and procedures like this:
$h-callPkgFunc(\%options, $pkg, $func_name, \$ret, @args);
$h-callPkgProc(\%options, $pkg, $proc_name, @parameters);
$pkg is the package name of synonym for the package.
Which package?
$func_name and $proc_name are the function or procedure name.
$ret is the return value from a function - which may be a reference
cursor for Oracle.
Could the call to callPkgFunc() just return a scalar, instead? I don't
mean can you change all of your code; I just mean, could the method
have been implemented that way?
@args is the list of scalar args for the function.
@parameters is the list of parameters for the procedure and if any
is a
reference to a scalar it is assumed to be an output parameter.
Oh, that's interesting. I like that.
There are various %options for whether to die etc and ways of
handling error output.
The wrapper handles creating the SQL, preparing it, binding the
parameters, executing the func/proc and returning the output bound
parameters.
Yeah, great!
This works well for us. We were using the same wrapper for MySQL and
DB2 but have since dropped use of MySQL and DB2. Of course, the
innards of the wrapper were significantly different between DB2,
MySQL and Oracle. For Oracle you end up with:
begin :1 := pkg_name.function_name(:2,:3,:4...); end;
begin pkg_name.proc_name(:1,:2,:3...); end;
Is that really the syntax for calling functions and procedures in
Oracle? Sheesh!
The code to do this is fairly straight forward, the complexities lie
in the differences between DBDs and databases.
It seems like it could be handled by the DBDs just setting a few
variables, though, I should think.
Maybe something like this:
sub call_proc {
my $dbh= shift;
my $name = shift;
my $opts = shift;
my $places = $dbh-create_places( [EMAIL PROTECTED], $opts );
return $dbh-selectrow_array(
$dbh-proc_sql( $name, $places, $opts ),
undef,
@_
);
}
sub call_func {
my $dbh= shift;
my $name = shift;
my $opts = shift;
my $places = $dbh-create_places( [EMAIL PROTECTED], $opts );
return $dbh-selectrow_array(
$dbh-func_sql( $name, $places, $opts ),
undef,
@_
);
}
sub create_places {
my ($self, $params, $opts) = @_;
join ', ', ('?') x @$params;
}
sub proc_sql {
my ($self, $proc, $places, $opts) = @_;
CALL $proc( $places );
}
sub func_sql {
my ($self, $func, $places, $opts) = @_;
SELECT $func( $places );
}
package DBD::Oracle;
sub create_places {
my ($self, $params, $opts) = @_;
# XXX Do whatever needs to be done for in/out args here.
join ', ', map { :$_ } ([EMAIL PROTECTED]);
}
sub proc_sql {
my ($self, $proc, $places, $opts) = @_;
die No package unless $opts-{package};
begin $opts-{package}.$proc( $places ); end;
}
sub func_sql {
my ($self, $func, $places, $opts) = @_;
die No package unless $opts-{package};
begin :1 := $opts-{package}.$func( $places ); end;;
}
Note how I've overridden the default driver methods for creating a
placeholder string for function and procedure arguments and SQL with
different ones in the DBD::Oracle driver.
I just banged this out as an example of where we might go, of course;
it's far from complete, and doesn't cover how to handle cursors or
result sets, of course. But just as an idea of the sort of thing I
think just might work.
A call-like method in DBI would save a little programming but for
some DBDs it would be difficult - I'm of course thinking of
DBD::ODBC. Although ODBC defines a {call xxx} syntax what actually
happens when you you use it is very database dependent and I even
know of ODBC drivers that expect you to ignore output bound
reference cursors in the parameter list.
Sure, for
Re: Unusual behavior with $sth-{NAME} and friends
On Aug 7, 2008, at 09:29, Greg Sabino Mullane wrote: For DBD::Pg, it prints: $VAR1 = [ 'Goldfish' ]; $VAR1 = [ 'goldfish' ]; $VAR1 = undef; $VAR1 = [ 'goldfish' ]; $VAR1 = undef; DBD::SQLite: $VAR1 = [ 'Goldfish' ]; $VAR1 = [ 'goldfish' ]; $VAR1 = undef; $VAR1 = [ 'goldfish' ]; $VAR1 = undef; DBD::mysql: $VAR1 = [ 'Goldfish' ]; $VAR1 = [ 'goldfish' ]; $VAR1 = [ 'Goldfish' ]; $VAR1 = [ 'goldfish' ]; $VAR1 = [ 'GOLDFISH' ]; Best, David
Re: RFC: developing DBD::SQLite
On Mar 26, 2009, at 10:30 PM, Darren Duncan wrote: Hello, So, out of my un-paid projects, my promise to take over release management of DBD::SQLite (from the still incommunicado previous owner) has now come to the front of my queue (now that Set::Relation 0.9.0 is out), so I'm now starting to think about it in detail and get to work over the next week or two. In that vein, the first and only major design change I intend to make right from the start is to stop bundling the SQLite library with the DBI driver and so the driver will have that library as an external dependency. --1. Prefer a system-installed lib, but use the bundled lib if one cannot be found on the system. Don't make this harder for people to use. While one of the selling points of DBD::SQLite versus other DBI drivers in the past was that it came with everything you need, with the advent of a single file amalgamation library being provided standard from sqlite.org, as well as the increasing availability of the SQLite library as its own shared system library install, I figure it isn't too difficult now for users to either obtain the library separately or use the one that came with their system, or the DBD::SQLite installer could automatically download it similar to how some other projects download their dependencies (Rakudo Perl 6 can download Parrot for example); so I don't think the ease of use of DBD::SQLite is diminished significantly by it no longer bundling the SQLite library. Don't download it; a lot of times modules are installed where there is no access to the Net. And those libraries that download external dependencies never work very well (see Math::Pari). On the other side, there is a lot of benefit gained from not bundling. For one thing the size of the distribution as well as the source control is cut down significantly, since the DBI driver alone is orders of magnitude smaller. For another thing, occasional needs to update for compatibility aside, DBD::SQLite will always be up to date with the newest SQLite library; users just update the library and possibly recompile the DBD::SQLite they already have. And so DBD::SQLite won't need to be updated as often; it would just need updates to address incompatible changes to the SQLite library, or to fix bugs in or update features specific to DBD::SQLite itself. These are benefits to the developer of the module, not to the end user. I don't find them compelling. Another quasi-change is that DBD::SQLite will be designed to work specifically with the amalgamation version of the source files only, not the pre-amalgamated versions; I say quasi-change since Audrey Tang already did the work to convert DBD::SQLite to work this way, in the separate ::Amalgamation distro. Don't know anything about this. Compatibility-wise, my DBD::SQLite will endeavour to work with all versions of SQLite 3.y.z, though note that only 3.4.0 for which the amalgamation file was a distinct download on sqlite.org (and 3.3.14 or so was the first that amalgamation was a make target). Or more specifically, I only plan to test with the latest SQLite source library available at the time (3.6.11 currently), as well as probably whatever version comes with Mac OS X Leopard. Supporting older versions will happen as I get advocates or testers for them. I also won't explicitly drop support for any Perl or DBI versions that the current DBD::SQLite supports, but I only intend to test it with the latest DBI and Perl 5.8.x+. If you don't test it with other versions, how can you be sure that they're supported? I deal with this with pgTAP, BTW; I have to keep 5 separate PostgreSQL installations around to make sure it works with all of them. You could probably script it to test that it works with n versions of SQLite. That would obviously be an improvement over the current maintenance. A minor change is I will start out with using 3-part versions and have a dependency on version.pm 0.74, which is bundled with Perl 5.10.x and an easy install otherwise. Why? I see no benefit to this, and just imposes yet another inconvenience on users. Now a specific question for you: First assume the new DBD::SQLite can look in at least 3 places for a SQLite library to use, which are: 1. An amalgamation file that the user explicitly put in the distro directory (or that was similarly slipstreamed into a copy of the distro maybe by some OS package manager); 2. A SQLite system shared library that was installed either as part of the OS or later by a user; 3. Go and automatically fetch a copy of the latest amalgamation file from sqlite.org, similarly to how Rakudo Perl 6 can go fetch a copy of its Parrot dependency from the 'net. Now assuming that, changeable config options aside, there is an automatic default order that these alternate sources will be used by a hands-free CPAN/CPANPLUS/etc
Re: RFC: developing DBD::SQLite
On Mar 28, 2009, at 12:40 AM, Darren Duncan wrote: I'm not a pushover. It's more that I wasn't strongly opinionated on the matter in the first place and I was fishing; your response led to me realizing that a simpler plan of action was better (and less work for both me and others). (Less work)++ And that's the end of this thread I think. (Thread death)++ :-) Best, David
Savepoints
Tim et al.,
Anyone given any thought to an interface for savepoints? They're a
part of the SQL standard, and basically look like named
subtransactions. The SQL looks like this:
BEGIN;
INSERT INTO table1 VALUES (1);
SAVEPOINT my_savepoint;
INSERT INTO table1 VALUES (2);
ROLLBACK TO SAVEPOINT my_savepoint;
INSERT INTO table1 VALUES (3);
RELEASE SAVEPOINT my_savepoint;
COMMIT;
Compared to transactions, the statements look like this:
TRANSACTIONS | SAVEPOINTS
-
BEGIN;| SAVEPOINT :name;
COMMIT; | RELEASE :name;
ROLLBACK; | ROLLBACK TO :name;
Given these terms, I think that DBD::Pg takes the correct approach,
offering these functions:
pg_savepoint($name)
pg_release($name)
pg_rollback_to($name)
All you have to do is pass a name to them. I'd therefore propose that
the DBI adopt this API, offering these functions:
savepoint($name)
release($name)
rollback_to($name)
The would essentially work just like transactions in terms of error
handling and whatnot. The example might look like this:
$dbh−{RaiseError} = 1;
$dbh-begin_work;
eval {
foo(...)# do lots of work here
$dbh-savepoint('point1');
eval {
bar(...)# including inserts
baz(...)# and updates
};
if ($@) {
warn bar() and baz() failed because $@;
}
$dbh−commit; # commit the changes if we get this far
};
if ($@) {
warn Transaction aborted because $@;
# now rollback to undo the incomplete changes
# but do it in an eval{} as it may also fail
eval { $dbh−rollback };
# add other application on−error−clean−up code here
}
If the transaction succeeds but the savepoint fails, the foo() code
will be committed, but not bar() and baz().
Thoughts?
Best,
David
Re: Savepoints
On Sep 29, 2009, at 12:34 PM, David E. Wheeler wrote: Compared to transactions, the statements look like this: TRANSACTIONS | SAVEPOINTS - BEGIN;| SAVEPOINT :name; COMMIT; | RELEASE :name; ROLLBACK; | ROLLBACK TO :name; Apologies, I made a mistake. It should actually be: TRANSACTIONS | SAVEPOINTS - BEGIN;| SAVEPOINT :name; COMMIT; | RELEASE SAVEPOINT :name; ROLLBACK; | ROLLBACK TO SAVEPOINT :name; I still think that methods named savepoint(), release(), and rollback_to() are ideal, but I could also see an argument that the latter two be named release_savepoint() and rollback_to_savepoint(), though they're a bit long. Best, David
Re: Savepoints
On Sep 29, 2009, at 2:25 PM, Darren Duncan wrote: 1. The basic idea is that we have nested transactions, and starting a child is defining a subunit that needs to succeed or be a no-op as a whole. I agree in principal; DBIx::Class has this feature, and it's fairly nice. I'm borrowing it for my own project, which is what led me to send my post. I don't think there's a place for it in the DBI, however, since the DBI offers no other features that work in a nested manner. 2. DBI is always in autocommit mode by default, because that treats each SQL statement as an innermost nested transaction of its own. There should not be an autocommit=0 in the interest of consistency. Are you saying that a transaction not be required for savepoints? I'm pretty sure that a transaction is required for savepoints. Perhaps a call to savepoint() implicitly starts a transaction; not sure if that's a good idea. 3. A slightly higher level of abstraction would provide the greatest user-friendliness, and I strongly prefer the idea of sub- transactions being tied to a lexical scope or block, such as a try- block. So for example, entering a sub-transaction block starts a child transaction, exiting one normally commits that child, and exiting abnormally due to a thrown exception rolls back the child. I do, too, but the DBI probably should not work that way. You should be able to build that from the DBI. Making things scope-tied is the safest and easiest to use because users don't have to explicitly call commit/rollback for every begin, similar to how automatic memory management helps us not need to remember to do a 'free' for each 'malloc' in spite of the many ways a code block might be exited. In this situation, there would not be any explicit begin()/commit()/rollback() methods, and also the SQL itself can't call those unpaired. As for implementation, well I think there is a Perl module that implements sentinel objects or whatever they're called, which could be looked at for ideas. The reason I wouldn't want block-scoped subtransaction support in the DBI is because its current transaction interface does not work that way. Maybe it makes sense for transactions and savepoints to be implemented with block scoping in DBI 2, but not, I suspect, DBI 1. 4. Less ideal for users, but perhaps closer to bare metal or what people are used to, DBI can keep its existing start/begin()/commit()/ rollback() methods, and they just get reused for child transactions. There should be a transaction nesting level counter which DBI exposes with a getter method. When a connection starts, the level is 0. Starting a transaction increments this by 1, and ending (commit or rollback) decrements it; decrementing it below zero is an error. The start/begin() method starts a new child transaction, or a first transaction if there are none, and commit()/rollback() ends the innermost transaction. Transactions Savepoints. This would be a mistake. This all said, if you still want to have actual named savepoints, well David's proposal sounds fairly decent. And lets folks like you and me build just what you describe on top of it, quite easily. Best, David
Re: Savepoints
On Sep 29, 2009, at 2:51 PM, Darren Duncan wrote: Going the other way, in SQL, there is a single terse SQL statement for starting/ending transactions, and doing the thing with savepoints. So for aside from maybe some consistency with legacy DBI features, why should DBI objects have begin/commit/rollback or methods specific to start/end savepoints at all? Why doesn't the user just do it in SQL like they do everything else in SQL? Its not like DBI is abstracting away other SQL language details, so why should it do so with the transaction/savepoint managing SQL? Unless some DBMSs support transactions but not with SQL? So maybe changing nothing in DBI is actually the best approach concerning savepoints. We already have the example of DBI transaction support; savepoints are just an extension of that. And if different databases support different syntaxes, it'd be nice to have that abstracted into drivers. For example, MySQL has the RELEASE SAVEPOINT syntax, but it does not exist in Oracle, where the releasing of savepoints is automatic. It's best to have an abstract DBI interface for things like that, IMHO. Yeah, I know that the DBI doesn't cover everything in the SQL standard in this way; nor would I want it to. But as I said, the example of savepoints follows along from the transaction stuff almost identically. Best, David
Re: Savepoints
On Oct 5, 2009, at 5:01 AM, Tim Bunce wrote: We already have the example of DBI transaction support; savepoints are just an extension of that. Well, as you said earlier David, Transactions Savepoints. No, but they're closely related. Commit and rollback are methods partly to support drivers that emulate transactions n some way, partly to ensure client and server state stay in sync, and partly to allow, at least in theory, the use of transaction managers. The same would apply to savepoints, which are perhaps best thought of as subtransaction markers. I'd be interested if someone could do the research to list what databases support savepoints and what syntax they use for the main statements. DBIx::Class has done this for a lot of databases. Check out http://dev.catalyst.perl.org/repos/bast/DBIx-Class/0.08/trunk/lib/DBIx/Class/Storage/DBI/ A grep in there finds support in MySQL, PostgreSQL, MSSQL, and Oracle: MSSQL: SAVE TRANSACTION $name; ROLLBACK TRANSACTION $name; MySQL: SAVEPOINT $name; RELEASE SAVEPOINT $name; ROLLBACK TO SAVEPOINT $name; Oracle: SAVEPOINT $name; ROLLBACK TO SAVEPOINT $name; Pg: $dbh-pg_savepoint($name); $dbh-pg_release($name); $dbh-pg_rollback_to($name); DBD::Pg is the only one to offer an API for it in the database handle. It looks like it uses a protocol-level command to do the work, but the database itself supports the SQL-standard interface that's identical to what mysql supports. Oracle's implementation is standard, except for the lack of a release command, which is apparently implicit. MSSQL's interface also lacks a release, and the names for the commands are non-compliant. To me, this stuff is so closely related to transaction management that there ought to be a uniform interface for it in the DBI. If DBI2 doesn't have an API for transactions, then I wouldn't argue to add savepoints there. But it sure makes sense to me for the DBI. Best, David
_concat_hash_sorted()
Howdy Tim et al.,
I've just released [DBIx::Connector](http://search.cpan.org/perldoc?DBIx::Connector
) to the CPAN. It does connection caching and transaction management,
borrowing pages from `connect_cached()`, Apache::DBI, and DBIx::Class,
but usable in any of these environments. The transaction management is
similar to that in DBIx::Class, but also includes savepoint support
(hence my earlier post). Blog entry [here](http://www.justatheory.com/computers/programming/perl/modules/dbix-connector.html
).
Anyway, I'm caching the connections under a key I'm constructing just
like `connect_cached()` does:
my $key = do {
no warnings 'uninitialized';
# XXX Change in unlikely event the DBI changes this
function.
join !\001, @_[0..2], DBI::_concat_hash_sorted(
$_[3], =\001, ,\001, 0, 0
)
};
I thought this the simplest thing to do, but I'm wondering, Tim, if it
might be possible to expose this interface? It seems like it'd be
generally useful. Thoughts?
Thanks,
David
Re: Savepoints
On Oct 5, 2009, at 12:23 PM, Darren Duncan wrote: SQLite also has savepoints, since 3.6.8 around January. See http://sqlite.org/lang_savepoint.html for details. SQLite: SAVEPOINT $name RELEASE [SAVEPOINT] $name ROLLBACK [TRANSACTION] TO [SAVEPOINT] $name Adding that to DBIx::Class shouldn't be difficult. Oh, awesome. I added it to [DBIx::Connector](http://search.cpan.org/perldoc?DBIx::Connector ). Thanks! David
Re: _concat_hash_sorted()
On Oct 5, 2009, at 5:41 PM, Darren Duncan wrote: But from what you've described in your blog to be the state of affairs, I think that having a distinct DBIx::Connector module is a good idea, versus embedding that functionality in a larger DBI-using module. Yes, that's the idea. I've never been in a situation to use cached connections before, but this module looks like it could be a good default practice to use when using DBI, if it seems to make connection caching work more correctly. Unless there might be some agreement for DBI itself to use those semantics (but it hadn't already). I think that the DBI has tried to stay away from dealing with forking and threading, though all the special-casing for Apache::DBI must've been a PITA. I'm plan to use DBIx::Connector for all my projects going forward. Best, David
Re: Savepoints
On Oct 7, 2009, at 3:24 PM, Tim Bunce wrote: We'll go with savepoint($name), release_savepoint($name) and rollback_to_savepoint($name) as method names. Got a commit bit? No. David
Re: test DBD::SQLite 1.26_05 - foreign keys!
On Oct 14, 2009, at 11:39 PM, Darren Duncan wrote:
Also be sure to look at the section http://sqlite.org/foreignkeys.html#fk_enable
, because you have to enable a pragma on each connect to use the
foreign keys feature; it isn't yet on by default for backwards
compatibility purposes.
I suggest you add something to the documentation showing how to always
enable this using a callback. Something like
my $enable_fks = sub { shift-do('PRAGMA foreign_keys = ON') };
my $dbh = DBI-connect($dsn, '', '', {
Callbacks = { connect = $enable_fks, },
});
If the user uses connect_cached, she should use this instead:
Callbacks = { 'connect_cached.reused' = $enable_fks, },
Best,
David
Re: Time to Document Callbacks
On Oct 24, 2009, at 2:50 PM, Tim Bunce wrote:
Callbacks are handled by the method dispatcher so all method names are
valid (so don't bother trying to list them in the docs :)
Plus the two special cases for connect_cached: 'connect_cached.new'
and
'connect_cached.reused'. (There's also '*' but that's not really
recommended.)
Thanks!
Tim.
p.s. Might be worth skimming through the archives
http://tinyurl.com/yl582mt
Thanks. Following up on [this post](http://markmail.org/message/fus3dfauxs6yz6sv
), I wrote this code:
my $dbh = DBI-connect('dbi:Pg:dbname=bric', '', '', {
Callbacks = {
execute = sub { print Set in DBH\n; return; }
}
});
my $sth = $dbh-prepare('SELECT id FROM pref WHERE name = ?');
#$sth-{Callbacks}{execute} = sub { print Set in STH\n; return; };
$sth-execute('Time Zone');
It output nothing. When I uncommented that second-to-last line, it
output Set in STH. So it seems that a callback added to the dbh for
a statement method name does not end up getting passed on to the
statement handle. So I guess the Callbacks attribute is not passed on
to statement handles created for the database handle? Seems a shame…
Best,
David
Re: Time to Document Callbacks
On Oct 25, 2009, at 10:24 PM, David E. Wheeler wrote:
It output nothing. When I uncommented that second-to-last line, it
output Set in STH. So it seems that a callback added to the dbh
for a statement method name does not end up getting passed on to the
statement handle. So I guess the Callbacks attribute is not passed
on to statement handles created for the database handle? Seems a
shame…
One other thing: It's nice that the callbacks execute before the
method call, so that you can disable it by undefing $_. But it'd be
equally handle to have callbacks after the method call. For example,
I'd love to be able to create a callback on a statement handle to
convert a timestamp column to a DateTime object:
$sth-{PostCallbacks}{fetch} = sub {
my ($sth, $row) = @_;
$row-[3] = DateTime::Format::Pg-parse_datetime($row-[3]);
};
Is this something that's done at any level in the callbacks, or would
it have to be added? Thoughts?
Thanks,
David
Re: Time to Document Callbacks
On Oct 28, 2009, at 2:59 AM, Tim Bunce wrote: That looks nice. Are STHs the only things that are children? DBHs are children of DRHs (but I try to avoid talking about DRHs). Yes, let's pretend they don't exist here. Here's a deal: you write some tests for ChildCallbacks in t/70callbacks.t and I'll implement them. Deal. Oh, nice. Does that mean you'll have more time for the DBI? That's the hope. W00t. Yeah, looks pretty nice. Would we also be able to pass them to prepare()? No. Much as I'd like to change prepare() to take method attributes like connect I'm nervous of making that change. I'd happily support someone else doing the leg work though. Out of my league. I'm starting to fail to see the point of callbacks aside from connect(). :-( I'm probably being over-cautious. Most drivers use fetch() or fetchrow_arrayref() as the lowest-level calling method used by the other fetch* ad select* methods. So applying the same callback to both would work find in most cases. If I'm applying a callback to the fetch method, I expect it to execute when I actually call the fetch method on the STH to which I applied it. Reasonable, no? Is there some reason that wouldn't happen? If so, I'd call it a bug in the driver, frankly. Hrm. That'd be inconsistent with the way precallbacks work. Yeah, well, it was just a guess off the top of my head :) I'd need to think about it and look at what could would be involved and what would be effcient. I was thinking it would get the same arguments as the precallback, with an additional one that's a reference to the return value(s). Best, David
Re: Time to Document Callbacks
On Oct 28, 2009, at 10:26 AM, David E. Wheeler wrote: Here's a deal: you write some tests for ChildCallbacks in t/70callbacks.t and I'll implement them. Deal. Done in r13445. Best, David
Re: Time to Document Callbacks
Howdy, I've just committed r13835, which documents Callbacks. Yay! A few notes: * Tim, you earlier said: I could arrange for Callbacks to only apply to methods called by the applicationi, and not to 'nested calls'. That's a fairly major change after this length of time but given the limited use Callbacks have had, and the lack of documentation, it's not out of the question. The downside is that to intercept all fetched rows in a general way (like some kind of plugin utility module might want to do) you now have to add callbacks for 7 $sth fetch* methods and 6 $dbh select* methods. Is that really preferable? And I replied: Well, yes. But if there were 'fetch*' and 'select*' keys that could go to all of them at once, that would be cool, too. I don't know if you've changed anything here. Even earlier, you wrote: If you're applying a callback to the fetch method and then your code calls fetchrow_hashref, for example, how do you know if the driver's fetchrow_hashref() method calls fetch() or fetchrow_arrayref()? The fetch and fetchrow_arrayref method are typically aliased by the driver, but they're two separate methods from the DBI dispatcher's point of view. Applying the same callback to both is a reasonable approach. I'm not sure what you decided to do about this, either. But I do think that whatever side-effects of such dispatch there are should be carefully documented as well. * I tried to include some useful examples, but one is DBD::Pg-specific and another is DBD::mysql-specific. Hope that's okay. * I didn't mention anything about the future possibility of post-method-call callbacks or an OO interface for setting up callbacks. I look forward to your edits and comments, and let me know if I can help with anything else for this release. Best, David
Re: Time to Document Callbacks
On Mar 6, 2010, at 3:45 PM, Tim Bunce wrote:
I was tempted to leave The cool thing is but opted to drop it as it
doesn't match the tone of the rest of the docs - although they are
rather dry :)
Pity. Thought I was starting to change that. ;-P
I've made assorted edits and added some extra info. A sanity check would
be most welcome.
Looks good, thanks. Pity you removed the `$dbh-{private_myapp_sql_mode}` bit,
though, as that's required when using Cconnect_cached(), which you almost
certainly are doing if you need this hack.
Best,
David
Re: Time to Document Callbacks
On Mar 7, 2010, at 5:43 AM, Tim Bunce wrote:
Looks good, thanks. Pity you removed the `$dbh-{private_myapp_sql_mode}`
bit, though, as that's required when using Cconnect_cached(), which
you almost certainly are doing if you need this hack.
Are you sure it's required when using connected()? The connected method
is only called for new connections.
Yes, I just verified it with Bricolage, which uses connect_cached. connected()
is called every time, whether or not a connection is a new connection.
BTW, here's another issue I forgot to mention. I installed the DBI from svn and
now get this error unless I rebuild each driver:
[Sun Mar 07 10:22:24 2010] [error] DBI/DBD internal version mismatch (DBI is
v95/s208, DBD ./mysql.xsi expected v94/s208) you probably need to rebuild the
DBD driver (or possibly the DBI).
I've never had an issue with binary compatibility between the DBI and a DBD.
Did something change in this last build?
Best,
David
Re: Time to Document Callbacks
On Mar 7, 2010, at 3:27 PM, Tim Bunce wrote: Uh, yeah, I just looked at the code. Sometimes I confuse myself. I think that's a bug. I always intended connected() to be used as an on-new-physical-connection-established hook. Any objections to making it so? Not from me, but you might get some bug reports. Looking at the code I can see an issue with clone(): it'll clone using the same method (connect/connect_cached) as the handle that's being cloned. I guess I can document that as a feature :) Never even noticed clone() before. But yeah, that sounds like a decent feature, as long as connect_cached does not return the exact same handle, eh? That is, clone() should always create a second, separate handle. Yes, the additional hook for sql_type_cast_svpv. But I shouldn't have bumped DBISTATE_VERSION for just that - the change was binary compatible with old drivers. (Drivers that care can use the DBIXS_REVISION macro to check if sql_type_cast_svpv is available at compile time and check it's non-zero to check it's available at runtime.) Dunno what sql_type_cast_svpv is for, but glad it's not just me. Fixed in r13837. Thanks. So there might be some folks around with the dev release who will have v95, even though the final will be v94? Best, David
connecting()?
Fellow DBIers, I was just discussing writing a callback on connect() to change authentication (my correspondent wants to use realm files). But then I discovered to my disappointment that there is no support for callbacks on connect(). This makes sense, frankly, since the method is called before the callbacks are applied, and callbacks execute before the method, not afterward (at least for now). But I've found great use by adding callbacks on connected(), so that they execute just after a connection. So I was wondering how folks felt about the idea of adding a connecting() method that executes just before the DBI tries to connect to the database? I think this would be a nice complement to connecting(), and instantly make a simple interface for adding a pre-connection callback, too. Thoughts? Best, David
Re: connecting()?
On Apr 15, 2010, at 5:17 AM, Tim Bunce wrote:
$drh = DBI-install_driver($driver);
$dbh = $drh-connect(...);
Assuming we added a $dbh-connecting() method, how would it be called,
and how would the callback have been enabled prior to calling it?
In connect(), before $connect_meth, just as connected() is called in that
method after $connect_meth.
$dbh-connected(@orig_args);
unless ($dbh = $drh-$connect_meth($dsn, $user, $pass, $attr)) {
Why not register the callback on the driver:
DBI-install_driver('Pg')-{Callbacks}{connect} = ...;
That'll affect all subsequent connects using that driver and won't work
with proxy drivers like Gofer, which is why DBI docs discourage messing
with driver handles, but it may fit your needs.
It's not my needs. Someone is creating a CPAN module I think. His needs were
met by connected(), but I think that connecting() would be useful as well.
If we did that (and handled backwards compatibility somewhow)
then presumably new_child() could apply the Callbacks attribute
which would then fire on the subsequent $dbh-connect call.
That would be cool. I, alas, have a very short supply of tuits these days.
Best,
David
Re: connecting()?
On Apr 16, 2010, at 3:13 AM, Tim Bunce wrote:
In connect(), before $connect_meth, just as connected() is called in that
method after $connect_meth.
$dbh-connected(@orig_args);
unless ($dbh = $drh-$connect_meth($dsn, $user, $pass, $attr)) {
$dbh is undef before the $drh-$connect_meth call.
Oh. Well that'd have to change.
It's not my needs. Someone is creating a CPAN module I think. His
needs were met by connected(), but I think that connecting() would be
useful as well.
Perhaps but it sure seems like a low priority.
I don't the tuits to do it, I admit.
That would be cool. I, alas, have a very short supply of tuits these days.
Ditto.
Is that what's holding back 1.610?
Best,
David
Re: Take care with version numbers (eg DBD::Pg)
On Jul 8, 2010, at 8:31 AM, Tim Bunce wrote: FYI On Thu, Jul 8, 2010 at 3:48 AM, Nigel Horne n...@bandsman.co.uk wrote: ! DBD::Pg2.6 2.17.1 Let's review version number math: 2.6 = 2.60 2.17.1 = 2.017001 2.60 2.017001 Looks like it should have been 2.6.0: 2.6.0 = 2.006001 2.17.1 = 2.017001 2.006001 2.017001 Version number suck. And clearly, three-version numbers suck harder. Best, David
Re: Take care with version numbers (eg DBD::Pg)
On Jul 8, 2010, at 9:38 AM, Jens Rehsack wrote: Looks like it should have been 2.6.0: 2.6.0 = 2.006001 2.17.1 = 2.017001 2.006001 2.017001 Version number suck. And clearly, three-version numbers suck harder. I think, the best way out would be a hard consensus and CPAN reindex. The closest thing to a concensus is represented in David's blog post: http://www.dagolden.com/index.php/369/version-numbers-should-be-boring/ Fortunately, version number parsing is gradually becoming stricter. You can only use strictly-formatted version numbers in `use MODULE VERSION` and `package MODULE VERSION` (as of 5.12). It will be at least 10 years before loosely-parsed version numbers are deprecated, though. And perhaps not then. Besides which, if my META.yml says DBD::Pg: 2.6 It assumes decimal notation (2.600), not three-digit (2.006). It has no way to tell that I really mean the latter. This might break some things in the first way - but my experience as Perl module packager for pkgsrc packaging system is, that most authors do not react until things fail (and many of them do not react then). But this will touch the sacred cow of downward compatibility ... We can't have both. No, we can't, alas. Best, David
Re: Take care with version numbers (eg DBD::Pg)
On Jul 8, 2010, at 10:09 AM, Greg Sabino Mullane wrote: Perhaps it would be good if the mixing of two and three dot versions on the same check was treated as a severe error and caused an automatic FAIL report. I can't see a case where using both forms would ever be desired. In my META.yml, I'll use three-digit notation for modules that use it (DBD::Pg) and decimal for those that don't (DBI). It might be useful for the version validation code to complain if I specify decimal and the module uses three-digit (or vice versa). But then that would screw things up for modules that unfortunately changed their versioning algorithm. I would no longer be able to require DBD::Pg 1.49, for example, even thought that's perfectly valid. Best, David
Re: Take care with version numbers (eg DBD::Pg)
On Jul 8, 2010, at 10:46 AM, Greg Sabino Mullane wrote: But then that would screw things up for modules that unfortunately changed their versioning algorithm. I would no longer be able to require DBD::Pg 1.49, for example, even thought that's perfectly valid. Good point, but hopefully such changes are a very rare and momentous event (as was the case with DBD::Pg). Version 1.49 (the last of the two dot versions for those playing at home) is *severely* deprecated. One of the reasons DBD::Pg jumped to 2.0.0 was to prevent any version comparison confusion, as even Perl's wacky versioning tools cannot deny that 2 1. :) A lot of folks changed without any momentous reason. So this suggestion, frankly, is right out. Frankly, I consider even momentous reason dubious. Pick a version and stick to it. I myself maintain a module or two with hinky version numbering systems because I inherited them and see no benefit to changing. Best, David
Re: Take care with version numbers (eg DBD::Pg)
On Jul 8, 2010, at 3:29 PM, Tim Bunce wrote: My take on this, for the record, is to prefer two part numbers in the form X.YYY where YYY never has a trailing zero. And thus may be X.Y or X.YY as well. Short, sweet, simple. Yeah, I'm with you. All of my modules use this format. (Except Bricolage. Don't ask.) Tim. p.s. No one commented on the DBI going from 1.609 to 1.611 :) That's one louder, isn't it? David
Re: Take care with version numbers (eg DBD::Pg)
Underscores should be banned from version numbers. Full stop. Best, David On Jul 8, 2010, at 3:46 PM, Darren Duncan wrote: Tim Bunce wrote: My take on this, for the record, is to prefer two part numbers in the form X.YYY where YYY never has a trailing zero. Short, sweet, simple. Tim. p.s. No one commented on the DBI going from 1.609 to 1.611 :) You mean now? 1.611 came out on April 29th. Or did you mean the completely different 1.611_93? Confusing! And that points to an example of something else that should become common practice for numbers. Projects that have any version X.Y_Z should never also have a version X.Y for the same X plus Y. Instead, the Y should always increment when moving between a developer release and a production release. See how DBD::SQLite does things for an example that I think is better. This is also analogous to Perl's own versioning X.Y.Z scheme, where there are never developer and production releases with the same Y. Its much less confusing that way. It also avoids the confusion of relating 1.002003 to 1.002_003, say; are those the same version or different versions? So, if the next DBI release after the latest 1.611_93 is going to be a stable release, then keep the current plan for it to be 1.612. Then, when making a subsequent dev release, call it 1.613_1 or 1.613_001 or such. Does that not make more sense? -- Darren Duncan
Re: Take care with version numbers (eg DBD::Pg)
On Jul 12, 2010, at 8:54 PM, Greg Sabino Mullane wrote: Doesn't that mean perl -MDBI -e 'print $DBI::VERSION' cannot indicate if you've got a dev version or a final release? Yes, thank god. Because that's not where that information should be! David
Please Test: AutoInactiveDestroy
Fellow DBIers, With a bit of help from me, Tim added a new feature to the DBI yesterday: AutoInactiveDestroy. I wrote the test for it. The test, however, forks. We need to make sure that it doesn't die an ugly death on systems without fork (and on Win32, where it's emulated). If you have such a system, would you minding checking out the repository and building and testing DBI? svn co http://svn.perl.org/modules/dbi/trunk Thanks! David
Re: Please Test: AutoInactiveDestroy
On Jul 27, 2010, at 5:01 AM, John Scoles wrote: Which test is it exactly?? t/16destroy.t Ok here you go Windblows XP home SP3 Visual C++ 2003 here is the output from the nmake test Thanks! t/01basics.t ... ok t/02dbidrv.t ... ok t/03handle.t ... ok t/04mods.t . ok t/05concathash.t ... ok t/06attrs.t ok t/07kids.t . ok t/08keeperr.t .. ok t/09trace.t ok t/10examp.t ok t/11fetch.t ok t/12quote.t ok t/13taint.t ok t/14utf8.t . ok t/15array.t ok t/16destroy.t .. 1/20 Test has no driver CLONE() function so is unsafe threade d Tim, is that something we should add to that test driver? t/zvp_16destroy.t .. 1/20 Test has no driver CLONE() function so is unsafe threaded t/zvp_16destroy.t .. Failed 2/20 subtests Hrm, can you run that one verbose, please? Thanks, David
Re: Please Test: AutoInactiveDestroy
On Jul 27, 2010, at 9:42 AM, Martin J. Evans wrote: Test has no driver CLONE() function so is unsafe threaded FAILED tests 19-20 Failed 2/20 tests, 90.00% okay Failed Test Stat Wstat Total Fail List of Failed --- t\zvp_16destroy.t 202 19-20 Failed 1/1 test scripts. 2/20 subtests failed. Files=1, Tests=20, 1 wallclock secs ( 0.00 cusr + 0.00 csys = 0.00 CPU) Failed 1/1 test programs. 2/20 subtests failed. C:\perlbuild_xxx\dbi_trunk As a wild guess, I've just added a CLONE() function in r14289. Would you mind trying again? I'm around on #dbi (as mje) tomorrow if that helps. Might be around this evening also but I'm not I've got a Windows machine available tonight. As am I. I'm theory. Best, David
Re: Please Test: AutoInactiveDestroy
On Jul 27, 2010, at 11:17 AM, John Scoles wrote: David E. Wheeler wrote: now we are getting t/zvp_16destroy.t(Wstat: 0 Tests: 18 Failed: 0) Parse errors: Bad plan. You planned 20 tests but ran 18. t/zvxgp_16destroy.t (Wstat: 0 Tests: 18 Failed: 0) Parse errors: Bad plan. You planned 20 tests but ran 18. Files=171, Tests=6280, 223 wallclock secs ( 3.73 usr + 0.70 sys = 4.44 CPU) Result: FAIL Failed 2/171 test programs. 0/6280 subtests failed. NMAKE : U1077: Stop. Verbose? Best, David
Re: Please Test: AutoInactiveDestroy
, at 11:17 AM, John Scoles wrote: David E. Wheeler wrote: now we are getting t/zvp_16destroy.t(Wstat: 0 Tests: 18 Failed: 0) Parse errors: Bad plan. You planned 20 tests but ran 18. t/zvxgp_16destroy.t (Wstat: 0 Tests: 18 Failed: 0) Parse errors: Bad plan. You planned 20 tests but ran 18. Files=171, Tests=6280, 223 wallclock secs ( 3.73 usr + 0.70 sys = 4.44 CPU) Result: FAIL Failed 2/171 test programs. 0/6280 subtests failed. NMAKE : U1077: Stop.
Re: Please Test: AutoInactiveDestroy
This is beyond my knowledge, alas. Anyone have any ideas? Maybe the failure John Scoles got can provide some clues? http://www.nntp.perl.org/group/perl.dbi.dev/2010/07/msg6159.html Paging Windows experts and Mssr. Bunce! Thanks, David On Jul 27, 2010, at 1:25 PM, Cosimo Streppone wrote: In data 26 luglio 2010 alle ore 23:11:42, David E. Wheeler da...@kineticode.com ha scritto: [...] AutoInactiveDestroy. We need to make sure that it doesn't die an ugly death on systems without fork (and on Win32, where it's emulated). It seems to die an ugly death for me. I'm experiencing the same failures as Martin. DBI SVN updated to r14289. Test output === (It segfaults at test 19) d:\dev\src\dbi-svnprove -vb t/zvp_16destroy.t t/zvp_16destroy.t .. 1..20 ok 1 - use DBI; ok 2 - ... check DBD::Test::dr::imp_data_size to avoid typo ok 3 - ... check DBD::Test::db::imp_data_size to avoid typo ok 4 - Install test driver ok 5 - Connect to test driver ok 6 - Should start active ok 7 - Should be active in DESTROY ok 8 - Create with ActiveDestroy ok 9 - InactiveDestroy should be set ok 10 - Should start active ok 11 - Should not be active in DESTROY ok 12 - Create with AutoInactiveDestroy ok 13 - InactiveDestroy should be set ok 14 - Should start active ok 15 - Should be active in DESTROY ok 16 - Create with AutoInactiveDestroy again ok 17 - InactiveDestroy should be set ok 18 - Should start active Dubious, test returned 5 (wstat 1280, 0x500) Failed 2/20 subtests Test Summary Report --- t/zvp_16destroy.t (Wstat: 1280 Tests: 18 Failed: 0) Non-zero exit status: 5 Parse errors: Bad plan. You planned 20 tests but ran 18. Files=1, Tests=18, 3 wallclock secs ( 0.08 usr + 0.05 sys = 0.12 CPU) Result: FAIL Perl binary (Windows Vista 32 bit) == This is perl, v5.10.0 built for MSWin32-x86-multi-thread. Summary of my perl5 (revision 5 version 10 subversion 0) configuration: Platform: osname=MSWin32, osvers=4.0, archname=MSWin32-x86-multi-thread uname='' config_args='undef' hint=recommended, useposix=true, d_sigaction=undef useithreads=define, usemultiplicity=define useperlio=define, d_sfio=undef, uselargefiles=define, usesocks=undef use64bitint=undef, use64bitall=undef, uselongdouble=undef usemymalloc=n, bincompat5005=undef Compiler: cc='cl', ccflags ='-nologo -GF -W3 -MD -Zi -DNDEBUG -O1 -DWIN32 -D_CONSOLE -DNO_STRICT -DHAVE_DES_FCRYPT -D_CRT_SECU RE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_IMPLICIT_CONTEXT -DPERL_IMPLICIT_SYS -DUSE_PERLIO', optimize='-MD -Zi -DNDEBUG -O1', cppflags='-DWIN32' ccversion='15.00.21022.08', gccversion='', gccosandvers='' intsize=4, longsize=4, ptrsize=4, doublesize=8, byteorder=1234 d_longlong=undef, longlongsize=8, d_longdbl=define, longdblsize=10 ivtype='long', ivsize=4, nvtype='double', nvsize=8, Off_t='__int64', lseeksize=8 alignbytes=8, prototype=define Linker and Libraries: ld='link', ldflags ='-nologo -nodefaultlib -debug -opt:ref,icf -libpath:c:\dev\perl510\lib\CORE -machine:x86' libpth=c:\dev\msvc9\vc\lib libs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32.lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib perllibs= oldnames.lib kernel32.lib user32.lib gdi32.lib winspool.lib comdlg32.lib advapi32.lib shell32.lib ole32. lib oleaut32.lib netapi32.lib uuid.lib ws2_32.lib mpr.lib winmm.lib version.lib odbc32.lib odbccp32.lib msvcrt.lib libc=msvcrt.lib, so=dll, useshrplib=true, libperl=perl510.lib gnulibc_version='' Dynamic Linking: dlsrc=dl_win32.xs, dlext=dll, d_dlsymun=undef, ccdlflags=' ' cccdlflags=' ', lddlflags='-dll -nologo -nodefaultlib -debug -opt:ref,icf -libpath:c:\dev\perl510\lib\CORE -mach ine:x86' Characteristics of this binary (from libperl): Compile-time options: MULTIPLICITY PERL_DONT_CREATE_GVSV PERL_IMPLICIT_CONTEXT PERL_IMPLICIT_SYS PERL_MALLOC_WRAP PL_OP_SLAB_ALLOC USE_ITHREADS USE_LARGE_FILES USE_PERLIO Built under MSWin32 Compiled at Jan 30 2008 08:15:22 @INC: c:/dev/perl510/lib c:/dev/perl510/site/lib . -- Cosimo
Re: patch full of dark magic
On Aug 6, 2010, at 5:42 PM, Jan Dubois wrote:
Against my better judgment I couldn't prevent myself from peeking at
the magical patch. I don't understand what it is doing, and I don't
have any candies so sacrifice right now either.
However, I'm always suspicious when I see code like this:
if ($^O =~ /win/i) { #Win32 dark magic. It works, so don't change anything
It isn't clear if that branch should be executed for Cygwin or not (currently
it will, although the comment sounds like it shouldn't, given that Cygwin is
considered to be a separate platform from Win32). I prefer to always make
things
explicit, either
if ($^O eq MSWin32 || $^O eq cygwin) {
or
if ($^O eq MSWin32) {
Agreed. I suspect that cygwin uses actual forking, no?
Of course the It works, so don't change anything comment is another alarming
red flag: you should not make changes to code if you don't understand what the
changes are doing and/or if you can't explain why it does what it does. Just
because you are getting rid of a symptom doesn't mean you solved the real
issue.
I *think* it works the way it does because on Win32 it's not really a fork, but
a thread, and when you exit in a child thread, it exits the parent, too.
Correct?
If so, then the attached patch should work just as well. Alexandr, can you give
it a try?
Oh, and BTW, you're my hero for looking into this!
Best,
David
win32-destroy.patch
Description: Binary data
Re: patch full of dark magic
On Aug 7, 2010, at 2:07 AM, Cosimo Streppone wrote:
Hi DBI Win32 hackers,
David E. Wheeler wrote:
I *think* it works the way it does because on Win32 it's not really a fork,
but a thread, and when you exit in a child thread, it exits the parent, too.
Correct?
Sounds reasonable?
Alexandr's patch works fine for me. Passes all tests.
That's fantastic!
Does my patch work, as well?
However, I'd be willing to spend a couple hours if someone can
guide me into compiling a debugging version with MSVC, and
trying to find out why this segfaults.
My really *ignorant* wild guess is that exit just avoids the
troubles bits of DESTROY() that are causing segfaults
without actually solving the problem.
So the issue is that it segfaults when you exit in DESTROY (which is called
during global destruction)? If so, this simple script should segfault for you:
package Foo;
sub DESTROY { exit; }
my $foo = bless {}, 'Foo';
Or perhaps it's only in the child, in which case this would segfault:
package Foo;
sub DESTROY { exit; }
my $foo = bless {}, 'Foo';
my $pid fork();
if ($pid) {
# parent.
wait;
} else {
# child.
}
If so, then the attached patch should work just as well. Alexandr, can you
give it a try?
[win32-destroy.patch]
Yours fails for me with the following output:
ok 18 - Should start active
not ok 19 - Child should be inactive on DESTROY# Failed test 'Child should
be inactive on DESTROY'
# at ./t/16destroy.t line 104.
ok 19 - Child should be inactive on DESTROY
ok 20 - Should be active in DESTROY
Oops. Try unless to if in this bit, please:
fail 'Child should be inactive on DESTROY'
if $self-FETCH('Active');
Thanks!
David
Re: patch full of dark magic
On Aug 9, 2010, at 8:21 AM, Cosimo Streppone wrote:
package Foo;
sub DESTROY { exit; }
my $foo = bless {}, 'Foo';
I couldn't try this yet, but, I don't think this
would segfault.
What I *guess* happens is that the 'exit' will cause
the script to immediately exit, skipping the nasty
part that would otherwise segfault.
Which nasty part?
Oops. Try unless to if in this bit, please:
fail 'Child should be inactive on DESTROY'
if $self-FETCH('Active');
Will try and let you know.
Thanks!
David
Re: patch full of dark magic
On Aug 9, 2010, at 9:51 AM, Jan Dubois wrote: I tried to look into this on the weekend, but ran out of time. I didn't get *any* segfaults though, neither from 16destroy.t nor from any of the samples below. I *did* only get 18 passing tests instead of 20 from 16destroy.t though. Yes, I think we've fixed that. Hope to commit it today once I have confirmation of my patch. exit() should not prevent DESTROY methods and END blocks from being called, but I haven't checked what happens during global destruction. The way to exit without any further processing whatsoever is POSIX::_exit(). On Windows however this will terminate both parent and child in a forking scenario because they are threads in the same process, and POSIX::_exit() will terminate the whole process, not just the current thread. That seems to be what's happening. The current code calls exit() in the child thread, and that kills the parent, too, but DESTROY apparently isn't called. Which is odd. Will try and let you know. Can you let me know Perl version is actually giving you a segfault? I tried both 5.10.1 and 5.12.1 and neither crashed, just didn't print any results for tests 19 and 20. Some details in the previous thread here: groups.google.com/group/perl.dbi.dev/browse_thread/thread/f683b62a637aeb76 Cosimo was getting segfaults with: This is perl, v5.10.0 built for MSWin32-x86-multi-thread. Best, David
Re: patch full of dark magic
On Aug 9, 2010, at 1:06 PM, Cosimo Streppone wrote: Which nasty part? Ehm... scrap that :) Your patch with 'unless' reversed to 'if' works fine. Thanks! Committed in r14317. Just for clarity, here's the full DESTROY() of 16destroy.t: Yep, thanks. Will try and let you know. Thanks! Done. Passes all tests. This is perl 5.10.0 on MSWin32 + MSVC9 (cc='cl' and ccversion '15.00.21022.08') W00t! Tim, I think we're ready for another dev release, especially since I added this test to the MANIFEST. Best, David
Re: patch full of dark magic
On Aug 9, 2010, at 4:21 PM, Jan Dubois wrote: On Mon, 09 Aug 2010, David E. Wheeler wrote: Tim, I think we're ready for another dev release, especially since I added this test to the MANIFEST. Running tests on the latest ActivePerl shows 2 issues: 1) t/01basics.t fails to load DBD::Oracle (because I don't have the Oracle client libraries installed on the test machine). This produces a popup that the user has to explicitly dismiss. The test then passes without errors. Why does it try to load DBD::Oracle? The DBI has no dependency on any DBDs other than those it ships with. Best, David
Re: patch full of dark magic
On Aug 9, 2010, at 4:44 PM, Jan Dubois wrote: DBI-installed_versions will attempt to load *all* installed drivers. You have DBD::Oracle installed but no Oracle libraries? But I see now that that test is only executed if you are running from a GIT or SVN checkout, so maybe it doesn't really matter. If you want to add the safeguard for it, here is a tested patch (the untested version was missing one pair of parens): Seems reasonable, thanks. Committed in r14322. Best, David
Re: patch full of dark magic
On Aug 10, 2010, at 3:09 PM, Jan Dubois wrote: You have DBD::Oracle installed but no Oracle libraries? Yes, DBD::Oracle is included in ActivePerl, but it will of course only work if you install the client libraries as well. Ah, I see. David
Re: Any reasons not to release DBI 1.614?
On Aug 31, 2010, at 2:52 AM, Tim Bunce wrote: It's back in. I may remove it for 1.615 or, more likely, may leave it out and individual developers deal with failure reports on perl 5.13.3+/5.14. You may “remove it…or, more likely, leave it out”? Huh? David
Re: Any reasons not to release DBI 1.614?
On Aug 31, 2010, at 1:08 PM, Tim Bunce wrote: It's back in. I may remove it for 1.615 or, more likely, may leave it out and individual developers deal with failure reports on perl 5.13.3+/5.14. You may “remove it…or, more likely, leave it out”? Huh? Ug. I meant may restore it or, more likely, leave it out. But you said “It's back in,” so I don't think you can restore it if it's already restored. Confused, David
Warnings
Hey All, Just installed 1.614 on three of my boxes. Looks good, and thanks for getting it out! I did notice some warnings, though: /usr/local/bin/perl /usr/local/lib/perl5/5.12.2/ExtUtils/xsubpp -typemap /usr/local/lib/perl5/5.12.2/ExtUtils/typemap -typemap typemap DBI.xs DBI.xsc mv DBI.xsc DBI.c cc -c -fno-common -DPERL_DARWIN -no-cpp-precomp -fno-strict-aliasing -pipe -fstack-protector -I/usr/local/include -O3 -DVERSION=\1.614\ -DXS_VERSION=\1.614\ -I/usr/local/lib/perl5/5.12.2/darwin-thread-multi-2level/CORE -W -Wall -Wpointer-arith -Wbad-function-cast -Wno-comment -Wno-sign-compare -Wno-cast-qual -Wmissing-noreturn -Wno-unused-parameter DBI.c DBI.xs: In function ‘dbih_setup_handle’: DBI.xs:1280: warning: value computed is not used DBI.xs:1318: warning: value computed is not used DBI.xs:1319: warning: value computed is not used DBI.xs:1324: warning: value computed is not used DBI.xs:1327: warning: value computed is not used DBI.xs: In function ‘dbih_set_attr_k’: DBI.xs:2062: warning: value computed is not used DBI.xs: In function ‘dbih_get_attr_k’: DBI.xs:2157: warning: value computed is not used DBI.xs:2384: warning: value computed is not used DBI.xs: In function ‘_profile_next_node’: DBI.xs:2590: warning: value computed is not used DBI.xs: In function ‘XS_DBI_dispatch’: DBI.xs:3143: warning: value computed is not used DBI.xs:3222: warning: value computed is not used DBI.xs: In function ‘XS_DBI__new_handle’: DBI.xs:4243: warning: value computed is not used DBI.xs: In function ‘XS_DBD_st_fetchrow_hashref’: DBI.xs:4965: warning: value computed is not used New DBIx::Connector now on its way to CPAN, too. Thanks, David
Re: [PATCH] DBD::Pg configuration setup
On Oct 15, 2010, at 7:43 AM, H.Merijn Brand wrote:
Good enough for now.
And here's the patch. FWIW HP-UX 9 is so archaic that even /I/ do not
have access to it anymore :)
--8---
--- Makefile.PL.org 2010-04-07 22:52:55 +0200
+++ Makefile.PL 2010-10-15 16:35:35 +0200
@@ -211,19 +211,16 @@ my %opts =
PERL_MALLOC_OK = 1,
NEEDS_LINKING = 1,
NO_META= 1,
- NORECURS = 1,
+NORECURS = 1,
clean = { FILES = 'trace Pg.xsi README.testdatabase' },
- realclean = { FILES = 'dbdpg_test_database/' },
+realclean = { FILES = 'dbdpg_test_database/' },
+macro = {
+ LDFLAGS = -L$POSTGRES_LIB $Config{ldflags},
+ LDDLFLAGS = -L$POSTGRES_LIB $Config{lddlflags},
+ },
);
-if ($os eq 'hpux') {
-my $osvers = $Config{osvers};
-if ($osvers 10) {
-print Warning: Forced to build static not dynamic on $os
$osvers.\a\n;
-$opts{LINKTYPE} = 'static';
-}
-}
-elsif ($os =~ /Win32/) {
+if ($os =~ /Win32/) {
my $msdir = $POSTGRES_LIB;
$msdir =~ s{$}{/ms};
$opts{LIBS}[0] .= -L$msdir -lsecur32;
--8---
Seems like a pretty straight-forward patch. Happy to see one OS-specific hack
removed.
David
Add Unicode Support to the DBI
DBIers, tl;dr: I think it's time to add proper Unicode support to the DBI. What do you think it should look like? Background I've brought this up a time or two in the past, but a number of things have happened lately to make me think that it was again time: First, on the DBD::Pg list, we've been having a discussion about improving the DBD::Pg encoding interface. http://www.nntp.perl.org/group/perl.dbd.pg/2011/07/msg603.html That design discussion followed on the extended discussion in this bug report: https://rt.cpan.org/Ticket/Display.html?id=40199 Seems that the pg_enable_utf8 flag that's been in DBD::Pg for a long time is rather broken in a few ways. Notably, PostgreSQL sends *all* data back to clients in a single encoding -- even binary data (which is usually hex-encoded). So it made no sense to only decode certain columns. How to go about fixing it, though, and adding a useful interface, has proven a bit tricky. Then there was Tom Christiansen's StackOverflow comment: stackoverflow.com/questions/6162484/why-does-modern-perl-avoid-utf-8-by-default/6163129#6163129 This made me realize that Unicode handling is much trickier than I ever realized. But it also emphasized for me how important it is to do everything on can to do Unicode right. Tom followed up with a *lot* more detail in three OSCON presentations this year, all of which you can read here: http://98.245.80.27/tcpc/OSCON2011/index.html (You're likely gonna want to install the fonts linked at the bottom of that page before you read the presentations in HTML). And finally, I ran into an issue recently with Oracle, where we have an Oracle database that should have only UTF-8 data but some row values are actually in other encodings. This was a problem because I told DBD::Oracle that the encoding was Unicode, and it just blindly turned on the Perl utf8 flag. So I got broken data back from the database and then my app crashed when I tried to act on a string with the utf8 flag on but containing non-unicode bytes. I reported this issue in a DBD::Oracle bug report: https://rt.cpan.org/Public/Bug/Display.html?id=70819 But all this together leads me to believe that it's time to examine adding explicit Unicode support to the DBI. But it needs to be designed as carefully as possible to account for a few key points: * The API must be as straightforward as possible without sacrificing necessary flexibility. I think it should mostly stay out of users ways and have reasonable defaults. But it should be clear what each knob we offer does and how it affects things. Side-effects should be avoided. * Ability to enforce the correctness of encoding and decoding must be given priority. Perl has pretty specific ideas about is and is not Unicode, so we should respect that as much as possible. If that means encoding and decoding rather than just flipping the utf8 bit, then fine. * The performance impact must be kept as minimal as possible. So if we can get away with just flipping the UTF-8 bit on and off, it should be so. I'm not entirely clear on that, though, since Perl's internal representation, called utf8, is not the same thing as UTF-8. But if there's an efficient way to convert between the two, then it should be adopted. For other encodings, obviously a full encode/decode path must be followed. * Drivers must be able to adopt the API in a straight-forward way. That is to say, we need to make sure that the interface covers what most (all?) drivers need. Some, like DBD::Pg, can specify that only one encoding come back from the database. Maybe others (DBD::mysql) can have individual columns in different encodings? It needs to cover that case, too. * It must be able to give the drivers some flexibility. Where we can't account for everything that all drivers need forever, we should make it possible for them to add what they need without changing the overall API or the meaning of the interfaces provided by the DBI. I'm not at all clear what such an API should look like. Based on my extensive experience with DBD::Pg, a fair amount of experience with DBD::SQLite, and limited experience with DBD::Oracle and DBD::mysql, I'd say it'd be useful to have at least these knobs: 1. An attribute indicating the database encoding. This is the encoding one expects all data coming from the database to be in. When this is set, the DBI or the driver would decode incoming data to Perl's internal format and encode data sent to the database. 2. A fourth param to bind_param() to indicate the encoding in which to send column data to the database. Defaults to the database encoding. 3. A new parameter to prepare() to indicate the encodings of specific columns to be selected. 4. An ENCODING attribute on statement handles that indicates the encoding of each columns. This is just a preliminary proposal, but covers most of the basics, I think. (I'm sure I'm suggesting the wrong places for some things). It does
Re: Add Unicode Support to the DBI
DBI peeps, Sorry for the delayed response, I've been busy, looking to reply to this thread now. On Sep 9, 2011, at 8:06 PM, Greg Sabino Mullane wrote: One thing I see bandied about a lot is that Perl 5.14 is highly preferred. However, it's not clear exactly what the gains are and how bad 5.12 is compared to 5.14, how bad 5.10 is, how bad 5.8 is, etc. Right now 5.8 is the required minimum for DBI: should we consider bumping this? I know TC would be horrified to see us attempting to talk about Unicode support with a 5.8.1 requirement, but how much of that will affect database drivers? I have no idea myself. I think I'd just follow TC's recommendations here. DBI should stay compatible as far back as is reasonable without unduly affecting further development and improvement (not that there's much of that right now). So if proper encoding is important to you, use at least 5.12 and prefer 5.14. And if proper encoding is not important to you, well, it is, you just don't know it yet. Another aspect to think about that came up during some offline DBD::Pg talks was the need to support legacy scripts and legacy data. While the *correct* thing is to blaze forward and use Do Things Correctly everywhere, I think we at least need some prominent knobs so that we can maintain backwards compatiblity for existing scripts that expect a bunch of Latin1, or need the data to come back in the current, undecoded, un-utf8-flagged way. Agreed. I suspect the existing behavior should remain the default, with a knob to make it do things correctly, with perhaps a deprecation plan to turn on the correctly knob by default in a year or so. Best, David
Re: Add Unicode Support to the DBI
On Sep 10, 2011, at 7:44 AM, Lyle wrote: Right now 5.8 is the required minimum for DBI: should we consider bumping this? I know a lot of servers in the wild are still running RHEL5 and it's variants, which are stuck on 5.8 in the standard package management. The new RHEL6 only has 5.10... So at this time the impact of such change could be significant. Yes, which is why we can't just impose a solution on people. Best, David
Re: Add Unicode Support to the DBI
On Sep 10, 2011, at 3:08 AM, Martin J. Evans wrote: I'm not sure any change is required to DBI to support unicode. As far as I'm aware unicode already works with DBI if the DBDs do the right thing. Right, but the problem is that, IME, none of them do the right thing. As I said, I've submitted encoding-related bug reports for every DBD I've used in production code. And they all have different interfaces for tweaking things. If you stick to the rule that all data Perl receives must be decoded and all data Perl exports must be encoded it works (ignoring any issues in Perl itself). Er, was there supposed to be a , then … statement there? I bow to Tom's experience but I'm still not sure how that applies to DBI so long as the interface between the database and Perl always encodes and decodes then the issues Tom describes are all Perl ones - no? The trouble is that: 1. They don't always encode or decode 2. When they do, the tend to get subtle bits wrong 3. And they all have different interfaces and philosophies for doing so Surely Oracle should return the data encoded as you asked for it and if it did not Oracle is broken. I'd still like to see this case and then we can see if Oracle is broken and if there is a fix for it. Oh I don't doubt that Oracle is broken. In some places DBD::Oracle does sv_utf8_decode(scalar) or SvUTF8_on(scalar) (depending on your Perl) and in some places it just does SvUTF8_on(scalar). I believe the latter is much quicker as the data is not checked. Many people (myself included) are particularly interested in DBD::Oracle being fast and if all the occurrences were changed to decode I'd patch that out in my copy as I know the data I receive is UTF-8 encoded. IME It needs an assume Oracle is broken knob. That is, I should have the option to enface encoding and decoding, rather than just flipping SvUTF8. And I think that such an interface should be standardized in the DBI along with detailed information for driver authors how how to get it right. See above. I'd like the chance to go with speed and take the consequences rather than go with slower but know incorrect UTF-8 is spotted. And maybe that's the default. But I should be able to tell it to be pedantic when the data is known to be bad (see, for example data from an SQL_ASCII-encoded PostgreSQL database). I thought UTF-8 when used in Perl used the strict definition and utf-8 used Perl's looser definition - see http://search.cpan.org/~dankogai/Encode-2.44/Encode.pm#UTF-8_vs._utf8_vs._UTF8 That's right. So if I want to ensure that I'm getting strict encoding in my database, It needs to encode and decode, not simply flip SvUTF8. Don't DBDs do this now? I know the encoding of the data I receive in DBD::ODBC and decode it when I get it and encode it when I send it and I believe that is what DBD::Oracle does as well. There is one exception in ODBC for drivers which don't truly abide by ODBC spec and send 8 bit data back UTF-8 encoded (see later). There is no single API for configuring this in the DBI, and I argue there should be. I've spent a lot of effort getting unicode working in DBD::ODBC (for UNIX and with patches from Alexander Foken for Windows) which is implemented in an awkward fashion in ODBC. I'd like to hear from DBD authors what support they already have and how it is implemented so we can see what ground is already covered and where the problems were. DBD::Pg's approach is currently broken. Greg is working on fixing it, but for compatibility reasons the fix is non-trivial (an the API might be, too). In a perfect world DBD::Pg would just always do the right thing, as the database tells it what encodings to use when you connect (and *all* data is encoded as such, not just certain data types). But the world is not perfect, there's a lot of legacy stuff. Greg, care to add any other details? as I remain unconvinced a problem exists other than incorrectly coded DBDs. I'm happy to collate that information. As a start I'll describe the DBD::ODBC: 1. ODBC has 2 sets of APIs, SQLxxxA (each chr is 8 bits) and SQLxxxW (each chr is 16 bits and UCS-2). This is how Microsoft did it and yes I know that does not support all of unicode but code pages get involved too. 2. You select which API you are using with a macro when you compile your application so you cannot change your mind. You can in theory call SQLxxxA or SQLxxxW functions directly but if you use SQLxxx you get the A or W depending on what the macro is set to. Problem: DBD::ODBC has to built one way or the other. 3. When using the SQLxxxA functions you can still bind columns/parameters as wide characters but the ODBC driver needs to support this. 4. When using SQLxxxW functions all strings are expected in UCS-2. You can bind columns and parameters as whatever type you like but obviously if you bind a unicode column as SQLCHAR instead of SQLWCHAR you probably get the
Re: Add Unicode Support to the DBI
On Sep 21, 2011, at 1:52 PM, Greg Sabino Mullane wrote:
Since nobody has actally defined a specific interface yet, let me throw out a
straw man. It may look familiar :)
===
* $h-{unicode_flag}
If this is set on, data returned from the database is assumed to be UTF-8,
and
the utf8 flag will be set.
I assume you also mean to say that data sent *to* the database has the flag
turned off, yes?
DBDs will decode the data as needed.
I don't understand this sentence. If the flag is flipped, why will it decode?
If this is set off, the utf8 flag will never be set, and no decoding will be
done
on data coming back from the database.
What if the data coming back from the database is Big5 and I want to decode it?
If this is not set (undefined), the underlying DBD is responsible for doing
the
correct thing. In other words, the behaviour is undefined.
===
I don't think this will fit into DBD::Pgs current implementation perfectly,
as
we wouldn't want people to simply leave $h-{unicode_flag} on, as that would
force SQL_ASCII text to have utf8 flipped on. Perhaps we simply never, ever
allow that.
You mean never allow it to be flipped when the database encoding is SQL_ASCII?
Best,
David
Re: Database/DBD Bridging?
On Sep 21, 2011, at 7:53 PM, Brendan Byrd wrote: Okay, this is a big blue sky idea, but like all things open-source, it comes out of a need. I'm trying to merge together Excel (or CSV), Oracle, Fusion Tables, JSON, and SNMP for various data points and outputs. DBIC seems to work great for a large database with a bunch of tables, but what about a bunch of databases? I've searched and searched, and nobody seemed to have designed a DBD for multiple DBDs. There's DBD::Multi and Multiplex, but that's merely for replication. This would require reparsing of SQL statements. I think it'd be simpler to use something like SQL/MED. In PostgreSQL 9.1, for example, you can install foreign data wrappers so that you can create tables inside PostgreSQL that actually point to CSV, MySQL, Oracle, Redis, or any number of other sources. It's read-only right now, though that's often all people need. Some example FDWs: http://www.postgresql.org/docs/current/static/file-fdw.html http://pgxn.org/tag/fdw ISTR that SQL Server has had similar capabilities for a long time, and I know Access has. Hell, ODBC might get you a long way toward it. In short, I think that a multi-backend DBD is probably the wrong level at which to do this, waay too much work. Best, David
Re: Add Unicode Support to the DBI
On Sep 22, 2011, at 11:14 AM, Martin J. Evans wrote: Right. There needs to be a way to tell the DBI what encoding the server sends and expects to be sent. If it's not UTF-8, then the utf8_flag option is kind of useless. I think this was my point above, i.e., why utf8? databases accept and supply a number of encodings so why have a flag called utf8? are we going to have ucs2, utf16, utf32 flags as well. Surely, it makes more sense to have a flag where you can set the encoding in the same form Encode uses. Yes, I agreed with you. :-) Unless I'm mistaken as to what you refer to I believe that is a feature of the Oracle client libraries and not one of DBD::Oracle so there is little we can do about that. Sure you can. I set something via the DBI interface and the DBD sets the environment variable for the Oracle client libraries. So to try and move forward, we'd we talking about a flag or flags which say: 1 encode the data sent to the database like this (which could be nothing) 2 decode the data retrieved from the database like this (which could be nothing but if not nothing it could be using strict or loose for the UTF-8 and utf-8 case) 3 don't decode but use SvUTF8_on (a specific case since Perl uses that internally and a number of database return UTF-8) one that seems to work but I worry about. 4 do what the DBD thinks is best - whatever the behaviour is now? Yes. and what about when it conflicts with your locale/LANG? So what? and what about PERL_UNICODE flags, do they come into this? What are those? and what about when the DBD knows you are wrong because the database says it is returning data in encoding X but you ask for Y. Throw an exception or a warning. and for DBD::ODBC built for unicode API am I expected to try and decode UCS2 as x just because the flag tells me to and I know it will not work? Seems like it only applies to the ANSI API in DBD::ODBC where the data could be UTF-8 encoded in a few (possibly broken see http://www.martin-evans.me.uk/node/20#unicode) cases. If the user does something that makes no sense, tell them it makes no sense. Die if necessary. I still think it would help to name some specific cases per DBD of flags in use and why they exist: DBD::ODBC has a odbc_utf8_on flag to say that data returned by the database when using the ANSI APIs is UTF-8 encoded and currently it calls SvUTF8_on on that data (I've never used or verified it works myself but the person supplying the patch said it had a purpose with a particular Postgres based database). That's what the new DBD::Pg flag that Greg's working on does, too. Beyond that DBD::ODBC has no other flags as it knows in the unicode/wide APIs the data is UCS2 encoded and it checks it is valid when decoding it. Similarly when sending data to the database in the wide APIs it takes the Perl scalar and encodes it in UCS2. Yeah, ideally, by default, if the DBD knows the encoding used by the database, it should just DTRT. There are backward compatibility issues with that for DBD::Pg, though. So there probably should be a knob to say don't do any encoding or decoding at all, because a lot of older apps likely expect that. DBD::Oracle to my knowledge has no special flags; it just attempts to do the right thing but it favours speed so most data that is supposed to be UTF-8 encoded has SvUTF8_on set but in one case (error messages) it properly and strictly decodes the message so long as your Perl is recent enough else it uses SvUTF8_on. So, what are the other flags in use and what purpose do they fulfill. I think we could really just start with one flag, encoding. By default the DBD should just try to do the right thing. If encoding is set to :raw then it should do no encoding or decoding. If it's set to :utf8 it should just turn the flag on or off. If it's set to an actual encoding it should encode and decode. I think that would be a good start. Best, David
Re: Add Unicode Support to the DBI
On Sep 22, 2011, at 11:57 AM, Martin J. Evans wrote: ok except what the oracle client libraries accept does not match with Encode accepted strings so someone would have to come up with some sort of mapping between the two. Yes. That's one of the consequences of providing a single interface to multiple databases. and what about when it conflicts with your locale/LANG? So what? I'm not so sure this is a So what as Perl itself uses locale settings in some cases - just thought it needed mentioning for consideration. I'm not really concerned about locales at this point. I tend to leave collation, for example, up to the database. Right now I'm strictly concerned about encoding. and what about PERL_UNICODE flags, do they come into this? What are those? See http://perldoc.perl.org/perlrun.html In particular UTF-8 is the default PerlIO layer for input streams of which reading data from a database could be considered one? That'd be cool, but it's not currently implemented that way, obviously. DBI and PerlIO are completely independent AFAIK, and the DBI doesn't look like a file handle. ok, I'm thinking through the ramifications of this. To add to the list I see DBD::SQLite has |sqlite_unicode |strings coming from the database and passed to the collation function will be properly tagged with the utf8 flag; but this only works if the |sqlite_unicode| attribute is set before the first call to a perl collation sequence and The current FTS3 implementation in SQLite is far from complete with respect to utf8 handling : in particular, variable-length characters are not treated correctly by the builtin functions |offsets()| and |snippet()|. and DBD::CSV has f_encoding = utf8, DBD::mysql has mysql_enable_utf8 which apparently This attribute determines whether DBD::mysql should assume strings stored in the database are utf8. This feature defaults to off. I could not find any special flags for DBD::DB2. DBD::Sybase has syb_enable_utf8 If this attribute is set then DBD::Sybase will convert UNIVARCHAR, UNICHAR, and UNITEXT data to Perl's internal utf-8 encoding when they are retrieved. Updating a unicode column will cause Sybase to convert any incoming data from utf-8 to its internal utf-16 encoding. Yeah, so I think that can be generalized. Best, David
Re: Add Unicode Support to the DBI
On Oct 2, 2011, at 8:49 PM, Greg Sabino Mullane wrote:
DEW I assume you also mean to say that data sent *to* the database
DEW has the flag turned off, yes?
No: that is undefined. I don't see it as the DBDs job to massage data
going into the database. Or at least, I cannot imagine a DBI interface
for that.
Uh, say what? Just as I need to
binmode STDOUT, ':utf8';
Before sending stuff to STDOUT (that is, turn off the flag), I would expect
DBDs to do the same before sending data to the database. Unless, of course, it
just works.
DEW Yeah, maybe should be utf8_flag instead.
Yes, very bad example. Let's call it utf8. Forget 'unicode' entirely.
Yeah, better, though it' just perpetuates Perl's unfortunate use of the term
utf8 for internal string representation. Though I suppose that ship has
sunk already.
Yeah, that last one is the current Postgres plan. Which I think should
be best practice and a default DBI expectation.
Agreed.
DEW DBDs will decode the data as needed.
DEW I don't understand this sentence. If the flag is
DEW flipped, why will it decode?
Because it may still need to convert things. See the ODBC discussion.
Oh, so you're saying it will decode and encode between Perl's internal form and
UTF-8, rather than just flip the flag on and off?
GSM If this is set off, the utf8 flag will never be set, and no
GSM decoding will be done on data coming back from the database.
DEW What if the data coming back from the database
DEW is Big5 and I want to decode it?
Eh? You just asked above why would we ever decode it?
Yes, because you were only talking about utf8 and UTF-8, not any other
encodings. Unless I missed something. If the data coming back from the DB is
Big5, I may well want to have some way to decode it (and to encode it for write
statements).
DEW You mean never allow it to be flipped when the
DEW database encoding is SQL_ASCII?
Yes, basically. But perhaps it does not matter too much. SQL_ASCII
is such a bad idea anyway, I feel no need to coddle people using it. :)
+1
MJE So is the problem that sometimes a DBD does not know what to encode data
MJE being sent to the database or how/whether to decode data coming back
from
MJE the database? and if that is the case do we need some settings in DBI
MJE to tell a DBD?
I think that's one of the things that is being argued for, here.
Yes.
MJE I think this was my point above, i.e., why utf8? databases accept and
MJE supply a number of encodings so why have a flag called utf8? are we
MJE going to have ucs2, utf16, utf32 flags as well. Surely, it makes more
MJE sense to have a flag where you can set the encoding in the same form
MJE Encode uses.
Well, because utf-8 is pretty much a defacto encoding, or at least way, way
more popular than things like ucs2. Also, the Perl utf8 flag encourages
us to put everything into UTF-8.
Yeah, but again, that might be some reason to call it something else, like
perl_native or something. The fact that it happens to be UTF-8 should be
irrelevant. ER, except, I guess, you still have to know the encoding of the
database.
MJE and what about when the DBD knows you are wrong because the database
MJE says it is returning data in encoding X but you ask for Y.
I would assume that the DBD should attempt to convert it to Y if that
is what the user wants.
And throw exceptions as appropriate (encoding/decoding failure).
MJE (examples of DBD flags)
Almost all the examples from DBDs seem to be focusing on the SvUTF8 flag, so
perhaps we should start by focusing on that, or at least decoupling that
entirely from decoding? If we assume that the default DBI behavior, or more
specifically the default behavior for a random DBD someone picks up is
flip the flag on if the data is known to be UTF-8, then we can propose a
DBI attribute, call it utf8_flag, that has three states:
* 'A': the default, it means the DBD should do the best thing, which in most
cases means setting SvUTF8_on if the data coming back is UTF-8.
* 'B': (on). The DBD should make every effort to set SvUTF8_on for returned
data, even if it thinks it may not be UTF-8.
* 'C': (off). The DBD should not call SvUTF8_on, regardless of what it
thinks the data is.
I still prefer an encoding attribute that you can set as follows:
* undef: Default; same as your A.
* ':utf8': Same as your B:
* ':raw': Same as your C
* $encoding: Encode/decode to/from $encoding
I presume the other half would be an encoding, such that
$h-{encoding} would basically ask the DBD to make any returned
data into that encoding, by hook or by crook.
With an encoding attribute, you don't need the utf8_flag at all.
Best,
David
Re: Add Unicode Support to the DBI
On Oct 7, 2011, at 1:47 AM, Tim Bunce wrote: Perhaps we could carve out some time at LPW to sit together and try to progress this. That would be awesome you guys! D
Re: Add Unicode Support to the DBI
On Oct 13, 2011, at 6:03 AM, Greg Sabino Mullane wrote:
I think what I haven't said is that we should just use the same
names that Perl I/O uses. Er, well, for the :raw and :utf8
varieties I was, anyway. Perhaps we should adopt it wholesale,
so you'd use :encoding(UTF-8) instead of UTF-8.
That's pretty ugly. I don't think we need to adopt the I/O
convention, as there is no direct mapping anyway, it just
confuses the issue.
Sure. In that case, I'd say :utf8, :raw, or $encoding.
For DBD::Pg, at least, if client-encoding is set to Big5, then
you *have* to encode to send it to the database. Or change the
client encoding, of course.
Not sure I'm following this completely. Or rather, why this should
be the DBDs role.
By default, yes, the DBD should DTRT here. But I think there also ought to be a
way to tell it what to do.
How would one map things - just demand that
whatever is given must be a literal encoding the particular database
can understand?
I think we should standardize on the Perl IO names for these things.
Some databases may not support them all, of course.
Hm... I don't know enough about the various DB's encodings to see
how good an idea that is.
I assume that it's all over the map, so we should be as general as we can.
Specifying an encoding by name should cover everything.
No, I think just encoding, and utf8 would be invalid,
but :encoding(UTF-8) would not.
Again, ugh. Although a *little* less confusing when contrasting:
$dbh-{encoding} = ':encoding(utf-8)';
$dbh-{encoding} = 'utf8';
Yeah, or we can go with my original suggestion:
$dbh-{encoding} = 'UTF-8';
$dbh-{encoding} = ':utf8';
Well, I think we might have to have it with the pg_prefix until
this stuff is finalized here. Not sure, though.
That's my point - if we can get it finalized here, we can avoif the
pg_prefix entirely, rather than add it now and then deprecate it later.
Sure. I suspect this is going to take a while, though.
Best,
David
Re: Add Unicode Support to the DBI
On Oct 7, 2011, at 5:06 PM, David E. Wheeler wrote: Perhaps we could carve out some time at LPW to sit together and try to progress this. That would be awesome you guys! So gents, do you plan to do this a bit? Martin, do you have the data you wanted to collect on this? Thanks, David
Re: Add Unicode Support to the DBI
On Nov 4, 2011, at 1:39 AM, Martin J. Evans wrote: Sorry David, I've been snowed under. I will try very hard to publish the research I found this weekend. Awesome, thanks. Did you ever get any data from DBD::SQLite folks? I didn't think I was going to make LPW but it seems I will now - although it has cost me big time leaving it until the last minute. Your sacrifice is greatly appreciated. :-) Best, David
Re: Add Unicode Support to the DBI
On Nov 4, 2011, at 10:33 AM, Martin J. Evans wrote: Did you ever get any data from DBD::SQLite folks? Yes. I found a bug in the process and it was fixed but I have a working SQLite example. Oh, great. I'm only really missing DB2 but I have contacts for that on #dbix-class who I've just not yet poked. Cool, thanks for the update. Best, David
Re: Add Unicode Support to the DBI
On Nov 8, 2011, at 5:16 AM, Tim Bunce wrote:
1. Focus initially on categorising the capabilities of the databases.
Specifically separating those that understand character encodings
at one or more of column, table, schema, database level.
Answer the questions:
what Unicode support is this database capable of? [vague]
are particular column data types or attributes needed?
does the db have a session/connection encoding concept?
does the db support binary data types.
does the client api identify data encoding?
A table summarizing this kind of info would be of great value.
I think this is the most important kind of data we need to move
forward with this topic. I suspect we'll end up with a few clear
levels of unicode support by databases that we can then focus on
more clearly.
+1. Yes, this should make things pretty clear.
2. Try to make a data-driven common test script.
It should fetch the length of the stored value, something like:
CREATE TABLE t (c VARCHAR(10));
INSERT INTO t VALUES (?) = $sth-execute(\x{263A}) # simley
SELECT LENGTH(c), c FROM t
Fetching the LENGTH is important because it tells us if the DB is
treating the value as Unicode. The description of DBD::Unify, for
example, doesn't clarify if the db itself regards the stored value
as unicode or the underlying string of encoded bytes.
Also probably best to avoid latin characters for this, I'd use
something that always has a multi-byte encoding, like a simley face char.
And something that doesn't have a variant that uses combining characters, so
that the length should be consistent if it's treated as Unicode.
3. Focus on placeholders initially.
We can ponder utf8 in literal SQL later. That's a separate ball of mud.
(I'd also ignore unicode table/column/db names. It's a much lower
priority and may become clearer when other issues get resolved.)
+1, though good to know about. Just as important as placeholders, however, is
fetching data.
4. Tests could report local LANG / LC_ALL env var value
so when others report their results we'll have that context.
Thanks again. I've only given it a quick skim. I'll read it again before LPW.
Meanwhile, it would be great if people could contribute the info for #1.
Tim.
p.s. Using data_diff() http://search.cpan.org/~timb/DBI/DBI.pm#data_diff
would make the tests shorter.
my $sample_string = \x{263A};
...
print data_diff($sample_string, $returned_string);
Can this be turned into a complete script we can all just run?
Thanks,
David
Re: Any reason not to make a new DBI release?
On Jan 29, 2012, at 11:06 AM, Martin J. Evans wrote: I'd be happy with that as I'm looking forward to asking people to use DBI_TRACE=DBD. I've had no problems with the lastest releases and subversion trunk. I’d like to see a statement about Unicode support in the docs, and any recommendations for how drivers should behave. Best, David
Oracle ping
Fellow DBIers,
When I wrote DBIx::Connector, I borrowed this code from DBIx::Class to “work
around an issue”:
sub ping {
my ($self, $dbh) = @_;
eval {
local $dbh-{RaiseError} = 1;
$dbh-do('select 1 from dual');
};
return $@ ? 0 : 1;
}
The reason for this workaround is described in [this
comment](https://rt.cpan.org/Ticket/Display.html?id=47005#txn-808055) from
Peter Rabbitison:
DBD::Oracle has some shutdown state in which it will return 1 on ping as long
as the socket is still open. This however did not guarantee the server is any
longer in a state to execute
queries. So what happened was:
1) the weird state is reached
2) a txn_do takes place and fails on the first sql command
3) the code calls ping() and gets a connected reply
4) the txn_do is not retried
5) ...
6) users lose profit
So a few questions about this:
1. Was this issue reported somewhere?
2. If so, was it fixed or otherwise worked around?
3. And if it was fixed, in what version of DBD::Oracle?
Thanks,
David
Re: Oracle ping
On Nov 2, 2012, at 4:46 AM, John Scoles byter...@hotmail.com wrote: David if you have a repeatable test or set of conditions or recipe of how you get into this state where DBD::Oracle pings but cannot run queries my buddies over at Oracle would love to get it. I don’t, I only know what Peter Rabbitson told me in RT #47005. David
Re: Oracle ping
On Nov 5, 2012, at 5:52 AM, John Scoles byter...@hotmail.com wrote: Does it automatically reconnect? That certainly seems like the wrong thing to do, especially in light of the docs for connect_cached. got that from the MySql doc it can be set up to reconnect if ping fails http://dev.mysql.com/doc/connector-python/en/myconnpy_MySQLConnection_ping.html not didn't check the DBD driver to see if it does this though. DBIx::Connector kills that option. https://github.com/theory/dbix-connector/blob/master/lib/DBIx/Connector/Driver/mysql.pm#L8 Best, David
Re: Oracle ping
On Nov 5, 2012, at 8:57 AM, Greg Sabino Mullane g...@turnstep.com wrote: Ah, so Oracle still requires an actual table? Thought they might have joined the rest of us by now in allowing things like 'SELECT 2+2;' :) No, it always requires a FROM clause, which is typically `FROM dual` for stuff like this. Best, David
Re: Oracle ping
On Nov 5, 2012, at 7:59 AM, Martin J. Evans martin.ev...@easysoft.com wrote: Just to reiterate what I eventually said. I don't want ping changed in DBD::Oracle. All I was saying is you should only read anything useful into ping returning false and not true unless your only purpose is like Oracle's OCIPing which checks your connection is still there (although not usable for some things). I believe the connection caching only does anything when ping returns false. Hrm. Well I wonder if I *should* just use DBI-ping in DBIx::Connector for Oracle rather than `select 1 from dual`. Or perhaps make it an option to switch to the select statement. Thoughts? David
Re: Oracle ping
On Nov 9, 2012, at 4:13 AM, John Scoles byter...@hotmail.com wrote: Well I made that ping patch for DBIx::Connector a few months ago on one of my production boxes with no impact but then again I am not doing anything fancy (no rac raid etc). If it is critical to DBDx::Connector to ensure a query can be run then no, if you are going for 100% work alike betweeen all DBDs then no again and I am sure H.Merijn would like that. An option would be nice as well At this point, I'm inclined to leave it as is. Why? Because one of the advantages of DBIx::Connector is fixup mode. Use that mode, and ping will only be called by txn() and friends only in the event of a failure. Which means that its overhead goes away, for the most part, no matter how it's implemented. So yeah, it's slower, but use fixup mode and avoid it altogether. Best, David
Connected Callback Errors Ignored
DBIers,
Given this script:
use v5.18;
use warnings;
use utf8;
use DBI;
my $dbh = DBI-connect('dbi:SQLite:', '', '', {
PrintError = 0,
RaiseError = 1,
AutoCommit = 1,
Callbacks = {
connected = sub {
say 'connected';
$_[0]-do('SELECT 1 from foo');
return;
},
}
});
say 'go';
$dbh-do('SELECT 1 from foo');
The output is:
connected
go
DBD::SQLite::db do failed: no such table: foo at /Users/david/bin/try line
22.
That doesn't seem right to me. Shouldn't the connected callback throw an
exception? IOW, I would not expect go to be output.
Best,
David
Re: Connected Callback Errors Ignored
On Nov 14, 2013, at 3:47 AM, Tim Bunce tim.bu...@pobox.com wrote: So I'll take a guess that you're not using DBI 1.630. (And hope I'm right.) Yep, I am using 1.628. I'm wondering, though, if callbacks should not be considered internal calls, and the outer handle passed to them? D
Re: Connected Callback Errors Ignored
On Nov 14, 2013, at 9:17 AM, David E. Wheeler da...@justatheory.com wrote:
The error about foo not existing is gone, overridden by the second error
about bar missing. This can lead to hard-to-find bugs. What if the second
query depended on a condition set up by the first, and the first failed? As a
user, I would only see the second error, and think that the first statement
had succeeded. It would take me quite a while to figure out that it had not,
in fact, succeeded.
I’m also finding it means I can’t do error handling in the callback. I have to
do this to get it to ignore errors:
$dbh-do('SET search_path = ?', undef, 'foo');
$dbh-set_err(undef) if $dbh-err;
I feel a little dirty.
All of which is to say, your fix in 1.630 certainly improves the situation,
but since callbacks are really userland code, I think it would be beneficial
to change callbacks to run in an outer context, with the outer DBI handles
passed to them. Possible?
This would eliminate all of these problems, no?
Thanks,
David
Re: Connected Callback Errors Ignored
On Nov 15, 2013, at 3:12 AM, Tim Bunce tim.bu...@pobox.com wrote:
You'd need to write the callback code in the way you'd naturally write
it when not using RaiseError. Which typically means something like:
$dbh-do('SET search_path = ?', undef, 'foo')
or return;
That will prevent any further code from executing, but does not give me a
chance to handle the error.
I’m also finding it means I can’t do error handling in the callback.
I'm not sure what you mean there.
I *always* use RaiseError = 1 (or HandleError). Never ever check return values.
I have to do this to get it to ignore errors:
$dbh-do('SET search_path = ?', undef, 'foo');
$dbh-set_err(undef) if $dbh-err;
I feel a little dirty.
Ignoring errors isn't a common action.
Oh, I agree, this is an edge case in that sense. The issue for me is more the
inconsistency. I have RaiseError set, but I cannot actually catch the error and
handle it where it happens, even though I have RaiseError set to true. I have
to know that callbacks are different, and either check return values to handle
errors (and call set_err(undef) if I have handled them in the callback), or
check return values and return (on 1.630 or later) to let an external error
handler catch the error. IOW, It’s a userland function but I have to write it
like an internal DBI function. It’s inconsistent UE.
Probably. There are three main down-sides to consider:
1. compatibility - the risk of breaking some existing callback code
Seems unlikely, I think, since it’s not documented that it’s not the user’s
handle that is passed to callbacks. I think I have been using callbacks longer
than anyone and never noticed. Frankly I’m amazed at how few people know
they’re there (I regularly get requests to add them to DBIx::Connector).
2. performance - the cost of getting the outer handle
3. pure-perl compatibility
There is an undocumented DBI::_handles function that returns the inner
and outer handles for a given (inner or outer) handle. Perl code within
DBI.pm, such as the connect_cached callback logic, would need to call
that (or something like it) to get the outer handle to pass to the callback.
Yeah. That’s not a ton over overhead, is it? Is an additional method call
really likely to have much of a performance impact?
The DBI::_handles implementation in lib/DBI/PurePerl.pm doesn't support
getting the outer handle from an inner one. That's probably fixable by
adding a weakref.
I don't have a strong opinion on this.
I am strongly in favor of providing a consistent interface to users. Currently,
error-handling in callbacks is inconsistent when using RaiseError or
HandleError.
I can carve out a little time today to write some test cases if it’ll help.
Thanks,
David
Re: Connected Callback Errors Ignored
On Nov 15, 2013, at 10:24 AM, Tim Bunce tim.bu...@pobox.com wrote: When the method call (which fired the callback) returns, the error recorded on the handle will trigger RaiseError etc. Only on 1.630 and higher. I *always* use RaiseError = 1 (or HandleError). Never ever check return values. Ah, your use of HandleError adds an extra perspective on this. Yeah, I have been defaulting to it for so long that I forget that some folks might not. Probably not. I think connect_cached is the only use of Callbacks in DBI.pm (not DBI.xs), and the very small added cost would only be paid by those using Callbacks anyway. And only on connection or reconnection, yes? Not like for every call to do() or fetch()! I can carve out a little time today to write some test cases if it’ll help. No need, I've just hacked t/70callbacks.t to include those tests :) Cool. Not pushed yet, I guess. Will watch for it! Thanks! David
Re: [perl5-dbi/dbi] b80de3: Make DBI::_handles return only the outer handle in...
On Nov 15, 2013, at 4:13 PM, Tim Bunce tim.bu...@pobox.com wrote: Make DBI::_handles return only the outer handle in scalar context Tim++ David
RFI: Database URIs
DBI Folks Gisle, I want to add support for specifying database connections as URIs to Sqitch, my DB change management system. I started working on it today, following the examples of JDBC and PostgreSQL. Before I release, though, I’d like a bit of feedback on a couple of things. First, I'm using the database name as the scheme in the URL. Some examples: postgresql://user@localhost:5433/dbname sqlite:///path/to/foo.db This is to make it easy to tell one DB from another. But I'm wondering if I should follow the example of JDBC more closely, and prepend db: or something to them. More like DBI DSNs, too. However, it would require a bit more work, as URI.pm does not currently recognize multiple colon-delimited strings as scheme names AFAICT. :-( Next, I've added a bunch of URI subclasses for various database engines. I’m not to familiar with some of them, so if you see any listed here where the name (to be used in the scheme) and the default port is wrong, please let me know: https://github.com/theory/uri-db/blob/master/t/db.t#L9 Thanks! David PS: Is this something the DBI might want to use?
Re: RFI: Database URIs
On Nov 22, 2013, at 8:48 PM, Darren Duncan dar...@darrenduncan.net wrote: postgresql://user@localhost:5433/dbname sqlite:///path/to/foo.db By database name do you mean DBMS name? Because I'd say the database name is what's on the right-hand side of the //, not what's on the left. Yes, correct. Another thing I was going to say is, if you wanted some standardization, you should distinguish the parts that are necessary to connect to a database from parts that just select a default schema in the database for interacting with. Getting a bit OT from my original questions here. But anyway, I’m following the JDBC and PostgreSQL examples here, where, frankly, are almost identical to most other URLs. This is a very good thing. By that I mean, remember that a PostgreSQL database and a MySQL database aren't actually the same concept. A PostgreSQL DBMS server gives access to multiple disjoint databases where you must name one to connect, and then separately from that is the optional concept of the current schema that you can select. A MySQL DBMS server gives access to exactly 1 database, which you can connect to without specifying a database name, and selecting a current schema (what they call a database) is optional for using MySQL. While I agree that a PostgresSQL database and a MySQL database are not the same thing, the MySQL community still calls it a database and tends to think of it that way, and JDBC has set the precedent for relying on what the community calls a “database” to be the “database” part of the URL: http://dev.mysql.com/doc/refman/5.0/en/connector-j-reference-configuration-properties.html The reason I say this is that DBI's uri scheme uses the same syntax for both database even though how they're treated is actually very different. And you don't have to do the same. I don’t have to, but that ship has long since sailed, so I will treat them the same as the DBI and JDBC. Best, David
Re: RFI: Database URIs
On Nov 25, 2013, at 3:50 AM, Jens Rehsack rehs...@gmail.com wrote: DBI-connect($dsn, $user, $passwd, \%attr) 4th argument is wasted in your current proposal. Er, well, I failed to provide a complete set of examples. Here’s one from the PostgreSQL docs: postgresql://other@localhost/otherdb?connect_timeout=10application_name=myapp All the attributes are provided by the GET query string. Best, David
Re: RFI: Database URIs
On Nov 25, 2013, at 11:08 AM, Jens Rehsack rehs...@gmail.com wrote:
Let’s go - shoot:
# specify most possible flags via driver flags
$dbh = DBI-connect (dbi:CSV:, undef, undef, {
f_schema = undef,
f_dir= data,
f_dir_search = [],
f_ext= .csv/r,
f_lock = 2,
f_encoding = utf8,
csv_eol = \r\n,
csv_sep_char = ,,
csv_quote_char = '',
csv_escape_char = '',
csv_class= Text::CSV_XS,
csv_null = 1,
csv_tables = {
info = { f_file = info.csv }
},
RaiseError = 1,
PrintError = 1,
FetchHashKeyName = NAME_lc,
}) or die $DBI::errstr;
And keep in mind, csv_tables can be more complex and there’re other
attributes like it eg. in DBD::AnyData.
Well, how you would want to handle params would be up to you. No, they are not
great for hashes, but do-able.
db:csv?f_dir=dataf_dir_search=foof_dir_search=barf_ext=.csv/rf_lock=2f_encoding=utf8csv_eol=%0D%0Acsv_sep_char=,csv_quote_char=%22csv_escape_char=%22csv+class=Text::CSV_XScsv_null=1RaiseError=1PrintError=1FetchHashKeyName=NAME_lccsv_tables=%7B%22info%22%3A%20%7B%22f_file%22%3A%22info_csv%22%7D%7D
So yeah, one would need to do some sort of parsting of nested data (JSON in the
csv_tables example here), though arrays work okay (e.g., f_dir_search).
OTOH, can you specify all this stuff in a DSN parseable by parse_dsn(), either?
But my point is not to replace the connect() API, but to create a standard of
sorts for representing connection info in a URL, to make it easier to specify
how to connect to things on the command-line. Yeah, if you want to do
everything, it will require more work, but that would be up to the code that
handles each driver, which is to say the URI subclass for a particular DBMS.
But this discussion orthogonal to my original questions, which were:
* Should I use a hierarchical scheme like JDBC? I’m leaning heavily toward this
now, just need a decent prefix, I'm thinking dbms, e.g., dbms:csv:foo.
* Do I have the metadata wrong for any of the DBMSes I have so far added
support for? Right now that’s just the ports they listen on by default:
https://github.com/theory/uri-db/blob/master/t/db.t#L9
Best,
David
Re: RFI: Database URIs
On Nov 25, 2013, at 11:02 PM, Jens Rehsack rehs...@gmail.com wrote: db:csv?f_dir=dataf_dir_search=foof_dir_search=barf_ext=.csv/rf_lock=2f_encoding=utf8csv_eol=%0D%0Acsv_sep_char=,csv_quote_char=%22csv_escape_char=%22csv+class=Text::CSV_XScsv_null=1RaiseError=1PrintError=1FetchHashKeyName=NAME_lccsv_tables=%7B%22info%22%3A%20%7B%22f_file%22%3A%22info_csv%22%7D%7D Happy hacking, when you want type that on command line. DBD::CSV (and other Pure-Perl drivers) is designed for flexibility and quick setup, not for expensive configuration and long-term running. Actually, I take that back. I am only focused on having a URI format for the DSN, not the attribute hash --- at least for now. So I would revise that URI to just dbi:csv:. Unless SQLEngine was modified to support additional parameters in the DSN, all the other stuff would have to be passed via the attribute hash. You can’t - this is a clear point in DBI::DBD::SqlEngine at the moment. Is that something that might change in the future? I tend to think of the DSN parts as being specific to a connection and specifiable by an end user --- such as the database name, server, username, password, etc. I tend to think of the attribute hash as for the developer, such as transaction and error handling. They are also more specific to the language -- I don't need a PrintError option in my Java app; it has some other API. All I say to you: if you want DBI supporting that - keep in mind the whole requirements. If you just want to do another DBI extension, with limited coverage and usage - feel free. I don't know if I want the DBI to support it or not. I would like to get feedback on how best to represent database connection information in URIs, and what I might be overlooking. I am not sure what to do about attribute hashes, and so am ignoring them for now. But this discussion orthogonal to my original questions, which were: * Should I use a hierarchical scheme like JDBC? I’m leaning heavily toward this now, just need a decent prefix, I'm thinking dbms, e.g., dbms:csv:foo“. You can’t - because there is no „foo“ database for DBD::CSV at the moment. Conceptual issue. In current state of development, those pure-perl DBMS simulating drivers have no persistent metadata as mysql and postgresql have. Well, that's just a matter of how you define that. Maybe foo is the name of the directory holding CSV files, and the file names are table names. I realize this is not how DBD::CSV currently works, just pointing out that these things are a matter of definition and implementation. I think you miss my point - I don’t say that you’re wrong, I say you lack a lot of requirements. When you don’t intend to support those pure-perl drivers, you’re fine and please go ahead. When you intend full DBI driver support, DBDI (https://github.com/timbunce/DBDI) could enlighten you. Following that concept would allow you a migration to p6 one fine day ;) I don't intend to support any drivers at all. I want to define a common URI standard for connecting to database engines without regard to any programming language or API at all. Best, David
Re: RFI: Database URIs
On Nov 25, 2013, at 11:21 PM, H.Merijn Brand h.m.br...@xs4all.nl wrote:
As I always use 2. when writing scripts, mostly because I use quite a
few useful attributes in the 4th argument already, it is the most
logical place: easy to maintain, easy to read and easy to extend.
Also very much specific to the DBI and DBDs, and a useful place for you as the
developer to define how the DBI should interface with the database.
One of the huge disadvantages of putting everything in the DSN is the
you have to rethink if and when character like :, ;, , and ' should be
escaped and how and that values like undef are pretty darn impossible
(unless the driver has taken the trouble to support that).
And as far as I can tell, the DBI does not currently support any escaping at
all in the DSN. I created a database with a comma in its name, and was unable
to figure out how to get the DBI to connect to it.
The first method however has a big advantage when the script (or a
one-liner) is completely driven by environment settings.
my $dbh = DBI-connect ($ENV{DBI_DSN});
which happens to be the (underdocumented) default for connect and thus
equivalent to
my $dbh = DBI-connect;
I dislike this, personally. As a developer, I *always* exercise a certain
amount of control over what gets passed in the attribute hash. It would be a
rare condition where I would want to let an end-user do it. A multi-database
command-line tool like dbish would be one of those rare exceptions.
Best,
David
Re: RFI: Database URIs
On Nov 26, 2013, at 12:42 AM, Tim Bunce tim.bu...@pobox.com wrote:
Why not define a direct translation from a URL to a DBI DSN?
A translation that doesn't require knowledge of any driver-specifics.
Because I want to the onus of connecting to the database to be on the
developer, not the end-user. I personally don't want to have to look up the
format of the URI for every database I connect to, and why the DBI DSN and JDBC
URL formats annoy me. I would rather have one solid, standard way to handle
most of the stuff needed for a connection string (host, port, username,
password, database name) and then pass additional stuff in a standard way (a
GET query string).
So I have implemented a `dbi_dsn` method on the database URI classes I’ve
written. The base class uses the most commonly-used format, which DBD::Pg, for
one, relies on:
$ perl-MURI -E 'say
URI-new(db:pg://me:sec...@example.com/foo.db)-dbi_dsn'
dbi:Pg:dbname=me:secret.com/foo.db
And then in subclasses I overrode appropriate methods to change the format.
Here’s Ingres, for example:
$ perl -MURI -E 'say
URI-new(db:ingres://me:sec...@example.com/foo.db)-dbi_dsn'
dbi:Ingres:foo.db
And here’s SQL Server:
$ perl -Ilib -MURI -E 'say
URI-new(db:sqlserver://me:sec...@example.com/foo.db)-dbi_dsn'
dbi:ODBC:Driver={SQL Server};Server=me:secret.com;Database=foo.db
Note that the URI is the same for all of these, except for the engine part of
the scheme. Additional Engine-specific parameters can be passed in the GET
string, and will simply be passed, through, e.g.,:
$ perl -Ilib -MURI -E 'say
URI-new(db:pg://me:sec...@example.com/foo.db??connect_timeout=10application_name=myapp)-dbi_dsn'
dbi:Pg:host=me:secret.com;dbname=foo.db;?connect_timeout=10;application_name=myapp
This puts the most of the onus on translating from the URL to the DBI DSN on
the developer of the URI class (me in this case) rather than on the user.
Best,
David
Re: RFI: Database URIs
On Nov 26, 2013, at 10:02 AM, David E. Wheeler da...@justatheory.com wrote: $ perl-MURI -E 'say URI-new(db:pg://me:sec...@example.com/foo.db)-dbi_dsn' dbi:Pg:dbname=me:secret.com/foo.db Well, I can see I have a bug or two to work out. That should be: $ perl -MURI -Ilib -E 'say URI-new(db:pg://me:sec...@example.com/foo.db)-dbi_dsn' dbi:Pg:host=example.com;dbname=foo.db Thanks, David
Re: RFI: Database URIs
On Nov 26, 2013, at 10:53 AM, David E. Wheeler da...@justatheory.com wrote:
Well, I can see I have a bug or two to work out. That should be:
$ perl -MURI -Ilib -E 'say
URI-new(db:pg://me:sec...@example.com/foo.db)-dbi_dsn'
dbi:Pg:host=example.com;dbname=foo.db
Oh silly me not escaping the @. Let’s try that again:
$ perl -Ilib -MURI -E 'say URI-new(shift)-dbi_dsn'
db:pg://me:sec...@example.com/foo_db
dbi:Pg:host=example.com;dbname=foo_db
$ perl -Ilib -MURI -E 'say URI-new(shift)-dbi_dsn'
db:ingres://me:sec...@example.com/foo_db
dbi:Ingres:foo_db
$ perl -Ilib -MURI -E 'say URI-new(shift)-dbi_dsn'
db:sqlserver://me:sec...@example.com/foo_db
dbi:ODBC:Driver={SQL Server};Server=example.com;Database=foo_db
Much saner. :-)
Best,
David
Re: RFI: Database URIs
On Nov 26, 2013, at 11:26 AM, Gisle Aas gi...@activestate.com wrote: I do find the db: prefix ugly. If you want users to see these strings I would think they find this prefix to be clutter too. Yeah. But I would thin, that if it *was* a standard, there would be just one scheme defined. That’s a guess, though. Maybe no one would care, and would not mind an unlimited number of new schemes (one for every db engine)? I don’t know. You seem to be alone in calling it pg:. For the other examples out there I see postgresql: or postgres:. Should all different ways be allowed and lead to the same thing? I've seen both mysql: and mysql2:. What about mariadb:? Yeah, I was going to add postgres and postgresql subclasses. Maybe pgsql, too, I dunno. Never seen mysql2; what’s that? Maria is the same as MySQL, really; I should add it, though, and have it inherit from db:mysql, just as vertica currently inherits from db:pg. For sqlite:-URLs people seem to disagree on how to specify relative vs absolute path names. Really? To me, an absolute path starts with /. Anything else is relative. I wish there actually was _a_ standard for this stuff. :-) I’m planning to write it up as a blog post and talk to some other communities about it. Thanks, David
Re: RFI: Database URIs
On Nov 26, 2013, at 11:49 AM, Gisle Aas gi...@activestate.com wrote: There is also precedence for using + in scheme names. Something like db+postgresql: then. It's still cluttered, and not really compatible with what other have used. Or x-postgres: while it's still experimental. Naming stuff is hard. Oh, I thought URI didn’t like +, but it turns out it is okay with it. $ perl -MURI -E 'say URI-new(x-ssh+foo://example.com)-scheme' x-ssh+foo Cool. Downside: Each DB engine requires its own scheme, which would make formal registration rather a PITA. I do see some registered “hierarchical” schemes using a dot, though, e.g., iris.beep, iris.xpc, etc.: http://www.iana.org/assignments/uri-schemes/uri-schemes.xhtml Surely someone has done this before, though. :-( Yeah, naming is hard. Probably why it’s one of my favorite things to do. :-) I don't know. Version 2 of the mysql protocol perhaps. I saw it at https://github.com/kennethreitz/dj-database-url/blob/master/dj_database_url.py#L17 Well, if it’s a real thing, I’m happy to add it. The scheme really should just be named after the protocol, not the kind of product you happen to find at the other end. But scheme != protocol. [Wikipedia](http://en.wikipedia.org/wiki/URI_scheme) says: URI schemes are frequently and incorrectly referred to as protocols, or specifically as URI protocols or URL protocols, since most were originally designed to be used with a particular protocol, and often have the same name. The http scheme, for instance, is generally used for interacting with web resources using HyperText Transfer Protocol. Today, URIs with that scheme are also used for other purposes, such as RDF resource identifiers and XML namespaces, that are not related to the protocol. Furthermore, some URI schemes are not associated with any specific protocol (e.g. file) and many others do not use the name of a protocol as their prefix (e.g. news). To me it makes sense not to tie it to a particular protocol. I want to connect to a database, and don’t much care about the protocol. The PostgreSQL libpq URL starts with postgresql://, not libpq://. Is sqlite:///path relative or absolute then? What about sqlite:/path? Both are absolute. sqlite://path and sqlite:path are both relative. This is how my implementation handles them. dj-database-url claims 4 slashes is the way to go; sqlite:path Wow, is that ugly. I was relying on the standard for file: URLs as the precedent. The ftp: standard had the same problem. It was once specified that ftp://server/%2Fpath was to be required to make the path absolute, while ftp://server/path was supposed to be relative to the home directory you ended up in after logging in to the ftp account. This was very confusing to users so most browers just implemented the path to always be absolute, with no way to access relative paths. That's at least how I remembered it. %2f is ugly. Of course, there is no host name in file: URIs. If we add it, as would be needed for Firebird, for example, I can see why dj-database-url ended up with four slashes: This is a full path: db:firebird://example.com//path/to/db So I guess these should be equivalent: db:firebird:path/to/db db:firebird:/path/to/db The first has an empty string for the network location, the second has no network location part at all. I’ve just committed a fix to support this. https://github.com/theory/uri-db/commit/cd60c7ac7e02572e5db3b39d2acb08b4a7fcfefe Best, David
Re: RFI: Database URIs
On Nov 26, 2013, at 12:43 PM, Tim Bunce tim.bu...@pobox.com wrote: ODBC complicates that further. Indeed. I want to avoid the protocol. I've now written up my proposal as a blog post: http://theory.so/rfc/2013/11/26/toward-a-database-uri-standard/ Thanks, David
Re: Why is selectrow_hashref complaining about a fetch without execute?
On Jul 16, 2015, at 6:40 AM, Tim Bunce tim.bu...@pobox.com wrote:
Well, this contains lots more light! ...
- dbd_st_execute for 03fdf4e0
parse_params statement
SELECT c.change_id ...
Binding parameters: SELECT c.change_id
-- do_error
Out of sort memory, consider increasing server sort buffer size error 1038
recorded: Out of sort memory, consider increasing server sort buffer size
-- do_error
- dbd_st_execute returning imp_sth-row_num 18446744073709551615
!! ERROR: 1038 'Out of sort memory, consider increasing server sort
buffer size' (err#0)
- execute= ( -1 ) [1 items] at /usr/lib/perl5/DBI.pm line 1632 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Role/DBIEngine.pm line 149
So execute failed. Note the crazy row_num. Execute seems to have
returned -1, which is a true value.
!! The ERROR '1038' was CLEARED by call to fetchrow_hashref method
- fetchrow_hashref for DBD::mysql::st
(DBI::st=HASH(0x42cfcc0)~0x4231cf8) thr#2603010
Then the higher-level code called fetchrow_hashref, which cleared the
error recorded by execute().
FWIW, the database handle is created like this:
my $dbh = DBI-connect($uri-dbi_dsn, scalar $self-username, $pass, {
PrintError = 0,
RaiseError = 0,
AutoCommit = 1,
mysql_enable_utf8= 1,
mysql_auto_reconnect = 0,
mysql_use_result = 0, # Prevent Commands out of sync error.
HandleError = sub {
my ($err, $dbh) = @_;
$@ = $err;
@_ = ($dbh-state || 'DEV' = $dbh-errstr);
goto hurl;
},
Context:
https://github.com/theory/sqitch/blob/master/lib/App/Sqitch/Engine/mysql.pm#L59
So I’m a little confused as to why the execute failure was ignored. Is this an
issue with DBD::mysql?
I assume the OP’s server could use some tuning. Seems pretty weird, though.
Thanks,
David
smime.p7s
Description: S/MIME cryptographic signature
Re: Why is selectrow_hashref complaining about a fetch without execute?
On Jul 14, 2015, at 3:24 AM, Tim Bunce tim.bu...@pobox.com wrote:
I can't see anything obvious from this trace. Come back with a level 4
trace and hopefully that'll shed sufficient light.
Here we go.
$ DBI_TRACE=4 sqitch status
DBI 1.630-ithread default trace level set to 0x0/4 (pid 10670 pi 2603010)
at DBI.pm line 288 via DBIEngine.pm line 7
# On database sqitch_testing
- DBI-connect(dbi:mysql:database=sqitch, root, , HASH(0x4231ae8))
- DBI-install_driver(mysql) for linux perl=5.018002 pid=10670 ruid=1000
euid=1000
install_driver: DBD::mysql version 4.031 loaded from
/usr/local/lib/perl/5.18.2/DBD/mysql.pm
- install_driver= DBI::dr=HASH(0x3341728)
!! The warn '0' was CLEARED by call to connect method
- connect for DBD::mysql::dr (DBI::dr=HASH(0x3341728)~0x33417e8
'database=sqitch' 'root' HASH(0x3f68f70)) thr#2603010
imp_dbh-connect: dsn = database=sqitch, uid = root, pwd = root
imp_dbh-my_login : dbname = sqitch, uid = root, pwd = root,host = NULL, port
= NULL
imp_dbh-mysql_dr_connect: host = |NULL|, port = 0, uid = root, pwd = root
imp_dbh-use_mysql_use_result: 0
imp_dbh-bind_type_guessing: 0
imp_dbh-use_server_side_prepare: 0
mysql_options: MYSQL_SET_CHARSET_NAME=utf8
imp_dbh-mysql_dr_connect: client_flags = 2
imp_dbh-mysql_dr_connect: -- connect= ( DBI::db=HASH(0x42cfc78) ) [1
items] at /usr/lib/perl5/DBI.pm line 671 via at (eval 231) line 58
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'RaiseError'
0) thr#2603010
STORE DBI::db=HASH(0x42cfc00) 'RaiseError' = 0
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 723 via at (eval
231) line 58
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'PrintError'
0) thr#2603010
STORE DBI::db=HASH(0x42cfc00) 'PrintError' = 0
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 723 via at (eval
231) line 58
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'AutoCommit'
1) thr#2603010
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 723 via at (eval
231) line 58
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER
'mysql_auto_reconnect' 0) thr#2603010
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'Username'
'root') thr#2603010
STORE DBI::db=HASH(0x42cfc00) 'Username' = 'root'
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'HandleError'
CODE(0x3ebbb10)) thr#2603010
STORE DBI::db=HASH(0x42cfc00) 'HandleError' = CODE(0x3ebbb10)
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER
'mysql_enable_utf8' 1) thr#2603010
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'Callbacks'
HASH(0x421a780)) thr#2603010
STORE DBI::db=HASH(0x42cfc00) 'Callbacks' = HASH(0x421a780)
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
- STORE for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER
'mysql_use_result' 0) thr#2603010
- STORE= ( 1 ) [1 items] at /usr/lib/perl5/DBI.pm line 726 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
{{ connected callback CODE(0x3ebbd08) being invoked
1 - do for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'SET SESSION
character_set_client = 'utf8'') thr#2603010
mysql.xs do() use_server_side_prepare 0, async 0
mysql_st_internal_execute MYSQL_VERSION_ID 50543
parse_params statement SET SESSION character_set_client = 'utf8'
1 - do= ( '0E0' ) [1 items] at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 75 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
1 - do for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'SET SESSION
character_set_server = 'utf8'') thr#2603010
mysql.xs do() use_server_side_prepare 0, async 0
mysql_st_internal_execute MYSQL_VERSION_ID 50543
parse_params statement SET SESSION character_set_server = 'utf8'
1 - do= ( '0E0' ) [1 items] at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 75 via at
/usr/local/share/perl/5.18.2/App/Sqitch/Engine/mysql.pm line 103
1 - do for DBD::mysql::db (DBI::db=HASH(0x42cfc00)~INNER 'SET SESSION
time_zone = '+00:00'') thr#2603010
mysql.xs do() use_server_side_prepare 0, async 0
mysql_st_internal_execute MYSQL_VERSION_ID 50543
parse_params statement SET SESSION time_zone
Re: DBD::mysql Re: Why is selectrow_hashref complaining about a fetch without execute?
On Jul 19, 2015, at 7:41 AM, Tim Bunce tim.bu...@pobox.com wrote: Internally the DBI has a DBIc_ROW_COUNT(sth) macro that has an IV type. That's a signed int that would be 64 bits on most modern systems. On many of those systems the plain int type might be 32 bits. I've just pushed an experimental change that might help in general https://github.com/perl5-dbi/dbi/commit/29f6b9b76e9c637be31cb80f1a262ff68b42ef43#diff-cb6af96fe009d6f8d9d682415e1ab755 but probably wouldn't in your case. Huh. Why not? At the moment I'd view this as a DBD::mysql bug. Tim. p.s. These open DBD::mysql issues might also be more or less related: https://rt.cpan.org/Public/Bug/Display.html?id=48158 https://rt.cpan.org/Public/Bug/Display.html?id=80394 https://rt.cpan.org/Public/Bug/Display.html?id=75570 Given that these have had exactly 0 activity in three years, how should we go about getting thins on the maintaners’ radar? Also, is there something I can do in Sqitch to work around this issue? Thanks, David
Why is selectrow_hashref complaining about a fetch without execute?
DBIers,
Got a complaint about a “fetch() without execute()” error. Asked for a trace,
got this. Looks like it’s coming from selectrow_hashref()? That shouldn’t
happen, right?
$ DBI_TRACE=1 sqitch status
DBI 1.630-ithread default trace level set to 0x0/1 (pid 3381 pi ed2010)
at DBI.pm line 288 via DBIEngine.pm line 7
# On database sqitch_testing
- DBI-connect(dbi:mysql:database=sqitch, root, , HASH(0x2b00a28))
- DBI-install_driver(mysql) for linux perl=5.018002 pid=3381 ruid=1000
euid=1000
install_driver: DBD::mysql version 4.031 loaded from
/usr/local/lib/perl/5.18.2/DBD/mysql.pm
- install_driver= DBI::dr=HASH(0x1c118e8)
!! The warn '0' was CLEARED by call to connect method
- connect('database=sqitch', 'root', ...)= ( DBI::db=HASH(0x2b9eac0) )
[1 items] at DBI.pm line 671
- STORE('RaiseError', 0)= ( 1 ) [1 items] at DBI.pm line 723
- STORE('PrintError', 0)= ( 1 ) [1 items] at DBI.pm line 723
- STORE('AutoCommit', 1)= ( 1 ) [1 items] at DBI.pm line 723
- STORE('HandleError', CODE(0x278aa80))= ( 1 ) [1 items] at DBI.pm line
726
- STORE('mysql_auto_reconnect', 0)= ( 1 ) [1 items] at DBI.pm line 726
- STORE('Callbacks', HASH(0x2ae94d0))= ( 1 ) [1 items] at DBI.pm line 726
- STORE('mysql_enable_utf8', 1)= ( 1 ) [1 items] at DBI.pm line 726
- STORE('mysql_use_result', 0)= ( 1 ) [1 items] at DBI.pm line 726
- STORE('Username', 'root')= ( 1 ) [1 items] at DBI.pm line 726
{{ connected callback CODE(0x278ac78) being invoked
}} connected callback CODE(0x278ac78) returned
- connected('dbi:mysql:database=sqitch', 'root', ...)= ( undef ) [1
items] at DBI.pm line 733
- connect= DBI::db=HASH(0x2b9eac0)
- STORE('dbi_connect_closure', CODE(0x1c11b10))= ( 1 ) [1 items] at
DBI.pm line 742
- FETCH('mysql_serverinfo')= ( '5.6.17-0ubuntu0.14.04.1-log' ) [1 items]
at mysql.pm line 106
- FETCH('mysql_serverversion')= ( '50617' ) [1 items] at mysql.pm line
109
!! The ERROR '1038' was CLEARED by call to fetchrow_hashref method
!! The ERROR '4' was CLEARED by call to fetch method
ERROR: 19 'fetch() without execute()' (err#0)
- DESTROY(DBI::st=HASH(0x2ae9638))= ( undef ) [1 items] at DBI.pm line
1639
!! ERROR: 19 'fetch() without execute()' (err#0)
- selectrow_hashref('
SELECT c.change_id
, c.script_hash
, c.change
, c.project
, c.note
, c.committer_name
, c.committer_email
, date_format(c.committed_at,
'year:%Y:month:%m:day:%d:hour:%H:minute:%i:second:%S:time_zone:UTC') AS
committed_at
, c.planner_name
, c.planner_email
, date_format(c.planned_at,
'year:%Y:month:%m:day:%d:hour:%H:minute:%i:second:%S:time_zone:UTC') AS
planned_at
, group_concat(t.tag SEPARATOR ' ') AS tags
FROM changes c
LEFT JOIN tags t ON c.change_id = t.change_id
WHERE c.project = ?
GROUP BY c.change_id
, c.script_hash
, c.change
, c.project
, c.note
, c.committer_name
, c.committer_email
, c.committed_at
, c.planner_name
, c.planner_email
, c.planned_a...', undef, ...)= ( undef ) [1 items] at
DBIEngine.pm line 116
- HandleError on DBI::db=HASH(0x2b9ea48) via CODE(0x278aa80) (undef)
- $DBI::state= 'S1000'
- $DBI::state= 'S1000'
- $DBI::state= 'S1000'
- $DBI::state= 'S1000'
- $DBI::state= 'S1000'
!! The ERROR '19' was CLEARED by call to selectcol_arrayref method
- DESTROY(DBI::st=HASH(0x2ae94b8))= ( undef ) [1 items] at mysql.pm line
218
- selectcol_arrayref('
SELECT COUNT(*)
FROM information_schema.tables
WHERE table_schema = ?
AND table_name = ?
', undef, ...)= ( [ '1' ] ) [1 items] at mysql.pm line 218
- DESTROY(DBI::db=HASH(0x2b9ea48))= ( undef ) [1 items] at Tiny.pm line
88
fetch() without execute()
- disconnect_all= ( ) [0 items] (not implemented) at DBI.pm line 750
! - DESTROY(DBI::dr=HASH(0x1c119a8))= ( undef ) [1 items] during global
destruction
Thanks,
David
smime.p7s
Description: S/MIME cryptographic signature
