Re: Cached statements hold strong references to bind values
On Fri, Jan 25, 2013 at 10:17:11AM +, Tim Bunce wrote: Re https://rt.cpan.org/Public/Bug/Display.html?id=82942 The key question: is this a bug or a feature? Can you walk me through your thought process on what makes this a feature? Not trying to be difficult, I am just having trouble seeing the use-case ;) Cheers
Re: Shared DBI driver test suite
On Wed, Jan 23, 2013 at 10:07:01AM +0100, Jens Rehsack wrote: On 22.01.13 23:42, Tim Bunce wrote: On Tue, Jan 15, 2013 at 09:47:02AM +0100, Jens Rehsack wrote: 1) having a lot of DBD::CSV tests (DBI related like ChopBlanks) moved or copied and maintained in DBI Think bigger. I do :P SQL::Statement can't rely on DBI, because it would create a circular dependency This on its own is not that much of a problem. DateTime can not really function without DateTime::TimeZone, and as such (DateTime) skips about 90% of its tests on a fresh install, with DateTime::TimeZone installing afterwards. Just pointing out that there is precedent, not that I particularly like it.
Re: Cached statements hold strong references to bind values
On Fri, Jan 25, 2013 at 10:59:46PM +, Tim Bunce wrote:
On Fri, Jan 25, 2013 at 09:40:07PM +1100, Peter Rabbitson wrote:
On Fri, Jan 25, 2013 at 10:17:11AM +, Tim Bunce wrote:
Re https://rt.cpan.org/Public/Bug/Display.html?id=82942
The key question: is this a bug or a feature?
Can you walk me through your thought process on what makes this a feature?
Not trying to be difficult, I am just having trouble seeing the use-case ;)
A cache statement is cached because the user asked for it to be cached.
Perhaps there is a mismatch of understanding here. For me the only
benefit of using a cached statement is to avoid a prepare. Executing the
*same* statement with the *same* values *without* rebind does not seem
to be possible in the first place (according to the docs at least):
`... If another call is made to prepare_cached with the same $statement
and %attr parameter values, then the corresponding cached $sth will be
returned without contacting the database server. ...`
$sth = $dbh-prepare_cached($sql);
I think it's reasonable to assume that binding a reference value to a
placeholder of a cached statement handle will cause the ref count of the
value to increase.
$sth-bind_param_inout(1, \@foo);
Yes, otherwise it may get lost before the consequent execute().
The particular case in the ticket uses an reference to an object with
stringfy magic:
$sth-bind_param(1, $object_that_stringifies);
which is an extra wrinkle.
All this seems like reasonable and natural behaviour to me.
However, it also seems potentially inconvenient, as you've found.
So, the question then is what options do we have for fixing it?
The most obvious thing to do would be to weaken the refs in
values %{ $sth-{ParamValues} }. That could be done in the
application, so a DBI change isn't strictly needed.
That won't work as per my above remark. Now that I think of it your
cached statements are not copies, they are extra refs. Which means that
you can't really weaken it in the cached ones only.
If it's agreed that the DBI should weaken the refs then
we'd need to discuss when. As soon as they're bound?
I can imagine cases where values are bound in a lexical scope
that's exited before the $sth-execute() call, so that's not safe.
After the execute()? No, execute might be called again.
If this is possible - docs/tests should reflect that (I am still fuzzy
on what the semantics are, hence no patches to offer).
After finish()? Maybe. The docs for finish is already say Calling finish ...
may also make some statement handle attributes (such as NAME and TYPE)
unavailable if they have not already been accessed (and thus cached).
It seems reasonable to extend that to say that any references in
ParamValues will be weaked.
After finish() won't work, because the app (DBIC in this case) doesn't
know to call finish() before garbage collection, which never happens
because of the circular ref. Augh!
Bound values are meant to persist across finish(), so there are still risks.
Also undocumented ;)
I'm not keen on adding another attribute just to control this edge case.
No, we shouldn't. After running through your thought process (thank you)
it is clear this can not be fixed with conventional approaches. I need
to wrap up Variable::Entangled that we specced out with Aristotle at
last YAPC::EU but then neither of us got to completion.
In the meantime stalling the bugreport with a link to this discussion
sounds sane.
[perl5-dbi/dbi] 366a7e: We want to verbosify cpanm before install commence...
Branch: refs/heads/dbi-test Home: https://github.com/perl5-dbi/dbi Commit: 366a7e6389e3514292d6aefc5ec53806ef2a1169 https://github.com/perl5-dbi/dbi/commit/366a7e6389e3514292d6aefc5ec53806ef2a1169 Author: Peter Rabbitson ribasu...@cpan.org Date: 2013-08-17 (Sat, 17 Aug 2013) Changed paths: M .travis.yml Log Message: --- We want to verbosify cpanm before install commences
ENORIBA (was: Weekly DBIT IRC meeting - Wed. 9:00 CET)
On Tue, Oct 01, 2013 at 08:18:56AM +0200, Jens Rehsack wrote: Hi all, Hi everyone! Since I am named repeatedly in the core group of devs responsible for DBIT, I figured I will publically announce I will not be able to get involved in the short term. At some point I took on a quick 2 week project. This was 2 months ago, and it is still going strong. Thus my OSS time is spread very thin, and will probably remain so for a couple months until I catch up on everything that got dropped. DBIT is one of the first victims, sorry Jens ;) I am still going to lurk around and am rather interested where you guys will end up. Also am glad Tim picked up the higher-level design bit - the project needed that. One more note about the design scope - I see DBIC interop being mentioned in multiple places as one of the goals. I would encourage the rest of you to design without it, as the test methodology of DBIC presents some rather unusual requirements, which will only serve to complicate the DBIT architecture. Down the road I may revisit what I could reuse/share, but until I have the time to sit down and think, the DBIC subgoal is just a hindrance. Cheers
[perl5-dbi/dbi] 284d1c: Fix test t/*35thrclone.t hang on older 5.8.9 per...
Branch: refs/heads/topic/concurrency_testfixes Home: https://github.com/perl5-dbi/dbi Commit: 284d1c78b6aa5fabe749cf2a21aaf0170e166ea6 https://github.com/perl5-dbi/dbi/commit/284d1c78b6aa5fabe749cf2a21aaf0170e166ea6 Author: Peter Rabbitson ribasu...@cpan.org Date: 2013-12-24 (Tue, 24 Dec 2013) Changed paths: M t/35thrclone.t Log Message: --- Fix test t/*35thrclone.t hang on older 5.8.9 perls Original test extension 3e2f434d did not properly copy the nanosleep code from the main loop below it Commit: 051ac72c319c66b40573742ddd9506af689b9017 https://github.com/perl5-dbi/dbi/commit/051ac72c319c66b40573742ddd9506af689b9017 Author: Peter Rabbitson ribasu...@cpan.org Date: 2013-12-24 (Tue, 24 Dec 2013) Changed paths: M t/10examp.t M t/19fhtrace.t Log Message: --- Fix t/*10examp.t and t/*19fhtrace.t failing under very high test parallelism Various versions of these tests end up working on the same file, add pid-based disambiguator Compare: https://github.com/perl5-dbi/dbi/compare/284d1c78b6aa^...051ac72c319c
Re: Bring AnyData / DBD::AnyData back to work with modern DBI
On 11/13/2014 07:12 AM, Jens Rehsack wrote: Reason to keep: People continuous ask me about AnyData / DBD::AnyData and send me fiddly patches hacking in the module, complain about working around compat issues etc. So what I currently know: every AnyData / DBD::AnyData user makes adoptions to the projects. On 11/13/2014 02:58 PM, Jens Rehsack wrote: Am 13.11.2014 um 14:37 schrieb Tim Bunce tim.bu...@pobox.com: My impression was that there was a risk of breaking existing users apps. If that can be avoided then great, and keeping the name would be best. Neither, nor :( They're already broken. everything is already broken contradicts people are using it and sending me patches. It can't be both ways It either is completely unused and nobody depends on it OR The module is heavily customized and worked around in tons of darkpan In the later case imnsho the module needs to be left alone. See Class::DBI for exactly this sort of case.
Re: Bring AnyData / DBD::AnyData back to work with modern DBI
On 11/13/2014 04:07 PM, Jens Rehsack wrote: ... political correctness guards (sorry, I do not see any advantage in the don't ever do this because it's forbidden in general). You need to clarify what you mean, language seems to be getting in the way. What do you perceive as political correctness, and what do you perceive as forbidden ?
Re: Accidently added CONV in SQL::Statement
On 02/05/2015 06:11 PM, Jens Rehsack wrote: Hi Michael, I think I will do. Math::Base::Convert with convert for up-to-base64 seems to be fine I still think providing an arbitrary up to base64 conversion is a mistake. Consider the standardized base32 alphabet: http://en.wikipedia.org/wiki/Base32
Re: DBD::mysql path forward
On 09/13/2017 09:58 PM, Dan Book wrote: On Wed, Sep 13, 2017 at 3:53 AM, Peter Rabbitson <rab...@rabbit.us <mailto:rab...@rabbit.us>> wrote: On 09/12/2017 07:12 PM, p...@cpan.org <mailto:p...@cpan.org> wrote: On Tuesday 12 September 2017 12:27:25 p...@cpan.org <mailto:p...@cpan.org> wrote: To prove fact that other DBI drivers (e.g. Pg or SQLite) had fixed similar/same UTF-8 issue as MySQL has and behave Perl-correctly, I would provide test cases so you would see difference between Pg, SQLite and mysql DBI drivers. And here is promised script: The script side-steps showcasing the treatment of BLOB/BYTEA columns, which was one of the main ( albeit not the only ) reason the userbase lost data. Please extend the script with a BLOB/BYTEA test. I'm not sure how to usefully make such a script, since correct insertion of BLOB data (binding with the SQL_BLOB type or similar) would work correctly both before and after the fix. If you were to actually try to make the requested addition you'd have noticed that DBD::Pg *DOES NOT* require one to binding with SQL_BLOB. For more info please go back to the old ( 2013 ) RT threads on the "mysql unicode issue" and read again carefully. Please try to actually implement this - it will give you additional perspective.
Re: DBD::mysql path forward
On 09/12/2017 07:12 PM, p...@cpan.org wrote: On Tuesday 12 September 2017 12:27:25 p...@cpan.org wrote: To prove fact that other DBI drivers (e.g. Pg or SQLite) had fixed similar/same UTF-8 issue as MySQL has and behave Perl-correctly, I would provide test cases so you would see difference between Pg, SQLite and mysql DBI drivers. And here is promised script: The script side-steps showcasing the treatment of BLOB/BYTEA columns, which was one of the main ( albeit not the only ) reason the userbase lost data. Please extend the script with a BLOB/BYTEA test.
Re: DBD::mysql path forward
On 11/09/2017 09:54 AM, p...@cpan.org wrote: As those people or projects misuse some internals of perl we cannot guarantee anything such that which may be broken or changed by updating any module from cpan or external source not related to DBD::mysql. Pali. This "argument" applies to a large portion of CPAN. All JSON variants, various web frameworks, even things as mundane as math libraries: https://rt.cpan.org/Public/Bug/Display.html?id=123268 As already stated more times, if there is some code which depends on some internals, it should stay conserved in tested version as it can break by updating anything (also unrelated). There is no other option how to achieve that code stay working and does not corrupt something else. Pali, this is *not* how CPAN works. Nor is this even how /usr/bin/perl works. Maintaining such thing is something what I believe nobody wants. For sure I'm not. I will state several very obvious ( to me ) things, apologies if they seem condescending: I simply do not know whether the participants on this thread are talking about the same thing. Pali's entire premise is flawed. In the grand scheme of things it makes absolutely no difference what "maintainers want". Zero. What the complaints in this thread are focused on is what the *users* want. And every maintainer, Pali included, are irrevocably bound by the general direction of wishes of the userbase. The wishes are roughly ( and in order of inmportance ): - Breakage is never silent and is rectified swiftly when reported ( and is ideally never argued about ) - Upgrades are noneventful as a whole, and are as modular as possible ( i.e. a DBD upgrade does not drag in Test2 or something similarly non-relevant ) - The overall library seems to move in a "better direction" ( more performance, less memory use, support for newer protocols, etc ) Satisfy the above - and you do get the privilege of being a maintainer of a central module with 15+ years of history. If one can't - then the user-base is better off with the module remaining "unmaintained" ( reusing Pali's terminology ), and Pali's brave-new-world plan ending up in a different namespace. DBD::MariaDB is an idea. Cheers
Re: DBD::mysql path forward
On 11/09/2017 01:46 PM, Noel Butler wrote: On 09/11/2017 21:32, p...@cpan.org wrote: What the complaints in this thread are focused on is what the *users* want. and the users want now, or will need in the near future, to build with latest, stable and recommended versions of MariaDB and MySQL. Which is a great thing to want. I want this too. This is not what the thread is about. The problem stems from Pali mashing together 3(!)* independent concerns, and presenting them as an "all or nothing" proposition: 1. Inability to build against latest libmysql libs 2. Several security fixes 3. Entirely redefining how unicode is handled throughout the stack If the ( artificial ) choice is the-above-3-together-or-nothing, then the logical answer is "of course we rather have nothing until someone else comes by and fixes 1 and 2, leaving 3 alone" If Pali is willing to *permanently* back off from changing anything related to 3 ( regardless how displeasing it is ) - then this becomes a completely different conversation. Not even a hint of such willingness has been demonstrated. This is what is being discussed. Cheers * There are actual 4 issues: there is also intentional breakage of bigint handling within the driver, which already shipped to CPAN at the start of 2017 and still has not been backed out, despite multiple pleas, including at least one large Perl shop declaring that DBD::mysql is unusable to them as it stands right now: https://github.com/perl5-dbi/DBD-mysql/pull/74
