Re: RaiseWarn attribute for DBI

[Dan Book]

As a side note, I have in the past thought a HandleWarn option may be
useful, for instance to log warnings from the database or other custom
behavior. It could also be used to throw exceptions.

-Dan

On Thu, Jan 17, 2019 at 4:03 AM  wrote:

> Hello!
>
> What do you think about adding a new attribute $dbh->{RaiseWarn} which
> cause that warnings reported by DBI drivers would behave like errors?
>
> For errors DBI has there $dbh->{PrintError} and $dbh->{RaiseError}
> attributes. First one is by default true and second one by default
> false. When PrintWarn is true, then all error from DBI driver are passed
> to perl's "warn" function and when RaiseError is true, then errors are
> passed to perl's "die" function. (Plus there is ability to register own
> error handler function)
>
> Currently DBI has only $dbh->{PrintWarn} attribute to control warnings.
> When is set to true (by default) all warnings from DBI driver are passed
> to perl's "warn" function.
>
> So I would propose to add $dbh->{RaiseWarn} attribute (off by default)
> to behave like $dbh->{RaiseError}, but for warnings.
>
> I have implemented this attribute and patch is there:
> https://github.com/perl5-dbi/dbi/pull/71/files
>

Re: RaiseWarn attribute for DBI

[Dan Book]

While I'm a staunch opponent of fatal warnings in general, and I believe
Pali has voiced concern with them as well, this is somewhat unrelated as it
has nothing to do with the core warnings pragma and categories, and is
simply an *option* to cause DBI to exhibit different behavior which it
currently cannot. As such I think it's a fine idea.

-Dan

On Thu, Jan 17, 2019 at 6:20 PM Alexander Hartmaier 
wrote:

> I didn‘t want to start a discussion about deprecation because I know the
> opinion about that for most Perl 5 developers.
>
> But strictures and its use in Moo showed that exceptions from warnings
> aren‘t welcome.
> You can install a warn handler in your code without requiring any change.
> Adding another option raises the number of combinations.
>
> As I don‘t see any benefit and no examples where given I‘m against it.
>
> > Am 17.01.2019 um 23:57 schrieb p...@cpan.org:
> >
> > Personally I do not like changing Print/Raise. It is documented,
> > implementation seems to match documentation, it is without bugs and
> > current behavior is usable.
> >
> > Anyway, back to my question about RaiseWarn. Do you think that it make
> > sense to have it in DBI?
> >
> >> On Thursday 17 January 2019 11:02:51 Darren Duncan wrote:
> >> Generally speaking, DBI is one of those things where backwards
> compatibility
> >> should be the highest concern after security.  There is a time and
> place to
> >> break compatibility, and this Print/Raise thing seems way too minor to
> me
> >> for that.  I support the version of this that is backwards-compatible
> and
> >> not the breaking version. -- Darren Duncan
> >>
> >>> On 2019-01-17 2:43 AM, p...@cpan.org wrote:
>  On Thursday 17 January 2019 11:06:22 Alexander Hartmaier wrote:
>  I'm aware of that, semantic versioning and major versions exist to
> handle API breakage.
> >>>
> >>> Such thing is unsupported by CPAN clients. So we cannot use it.
> >>>
> >>> Anyway, this is question for Tim as DBI maintainer. But I guess he does
> >>> not want to change API of DBI.
> >>>
>  My question is how to detect if an exception is because of a warn or
> a die when RaiseWarn is true.
> >>>
> >>> I guess you can use $dbh->err() method.
> >>> See: https://metacpan.org/pod/DBI#err
> >>>
> >>> A driver may return 0 from err() to indicate a warning condition after
> a
> >>> method call. Similarly, a driver may return an empty string to indicate
> >>> a 'success with information' condition. In both these cases the value
> is
> >>> false but not undef.
> >>>
> >>> Note that 'success with information' is not warning and therefore DBI's
> >>> PrintWarn or RaiseWarn ignores them.
> >>>
>  Best regards, Alex
> 
> > On 2019-01-17 10:53, p...@cpan.org wrote:
> > On Thursday 17 January 2019 10:23:25 Alexander Hartmaier wrote: > I
> don't see the benefit, Print* should die This would break existing API of
> DBI. Print just prints and Raise dies. This cannot be changed as there are
> many applications which depends on this API. > and I'd personally would
> release a major release and change the defaults as a breaking change:
> PrintError false, RaiseError true. > Can you name a use case and how to
> differ between an error and a warning at the error handling side? It is up
> to the DBI driver or database server what would result in is a warning and
> what in an error. > Best regards, Alex > > > On 2019-01-17 10:04,
> p...@cpan.org wrote: > > Hello! What do you think about adding a new
> attribute $dbh->{RaiseWarn} which cause that warnings reported by DBI
> drivers would behave like errors? For errors DBI has there
> $dbh->{PrintError} and $dbh->{RaiseError} attributes. First one is by
> default true and second one by default false. When PrintWarn is true, the
>   n all er
> 
>  ror from DBI driver are passed to perl's "warn" function and when
> RaiseError is true, then errors are passed to perl's "die" function. (Plus
> there is ability to register own error handler function) Currently DBI has
> only $dbh->{PrintWarn} attribute to control warnings. When is set to true
> (by default) all warnings from DBI driver are passed to perl's "warn"
> function. So I would propose to add $dbh->{RaiseWarn} attribute (off by
> default) to behave like $dbh->{RaiseError}, but for warnings. I have
> implemented this attribute and patch is there:
> https://github.com/perl5-dbi/dbi/pull/71/files
> 
> 
> >>>
> >>
>

Re: Bugtracker DBD::mysql

[Dan Book]

On Tue, Nov 14, 2017 at 8:34 AM, Michiel Beijen 
wrote:

> On Tue, Nov 14, 2017 at 11:07 AM,   wrote:
>
> > Hi! As there are only few tickets on github, it would be easier to
> > disable creating new tickets on github and those few which are not
> > resolved yet either move on RT or (if github allows it; which I believe
> > yet) continue to exists.
> >
> > And once any migration is ready, then it can be fully switched.
> >
> > Still one bugtracker is better then two, even if it is not on github.
>
> Yeah, I'm sure this is *easier*. But I really hate the RT cpan bug
> tracker, so I'd prefer to look again at migrating from RT >> Github -
> if that really would not work for some reason I'll consider migrating
> the other way around.
>
>
I also prefer github but of course it's mainly up to what is easiest for
the maintainers. Here is a script for migrating tickets if it's not what
you already tried: https://metacpan.org/pod/RTx::ToGitHub

-Dan

Re: DBD::mysql path forward

[Dan Book]

It seems to me like the remaining option that can make everyone "happy" is
the previously-suggested option of maintaining a legacy branch and doing
new development (reinstating 4.042) in another branch which will be
released as a new distribution, like DBD::mysql2, by the same maintainers.
(I would not favor DBD::MariaDB as a name, since this new distribution
would also be the favored way to connect to MySQL.) After doing so
DBD::mysql can be deprecated, with migration instructions added to the
docs, and only receive critical security fixes if anything. If done this
way the community should not be fractured; the current module will simply
be abandoned for future development, all support and maintenance will move
to the new one. Patrick and Michiel, what do you think?

-Dan

Re: DBD::mysql path forward

[Dan Book]

How can we proceed from here?
-Dan

On Mon, Sep 18, 2017 at 1:17 PM, Patrick M. Galbraith  wrote:

> Pali,
>
> Great! Now we can start moving forward.
>
> Sorry if my responses have been intermittent - first week at new job.
>
> Regards,
>
> Patrick
> On 9/16/17 4:35 AM, p...@cpan.org wrote:
>
> I prepared branch master-new, which is based on current DBD-mysql master
> branch and revert state to pre-4.043 version, including all changes done
> after 4.043 release to master branch. I have this master-new branch in
> my fork. If you want you can use it...
> https://github.com/pali/DBD-mysql/tree/master-new
>
>

Re: DBD::mysql path forward

[Dan Book]

On Wed, Sep 13, 2017 at 3:53 AM, Peter Rabbitson  wrote:

> On 09/12/2017 07:12 PM, p...@cpan.org wrote:
>
>> On Tuesday 12 September 2017 12:27:25 p...@cpan.org wrote:
>>
>>> To prove fact that other DBI drivers (e.g. Pg or SQLite) had fixed
>>> similar/same UTF-8 issue as MySQL has and behave Perl-correctly, I
>>> would provide test cases so you would see difference between Pg,
>>> SQLite and mysql DBI drivers.
>>>
>>
>> And here is promised script:
>>
>>
> 
>
> The script side-steps showcasing the treatment of BLOB/BYTEA columns,
> which was one of the main ( albeit not the only ) reason the userbase lost
> data.
>
> Please extend the script with a BLOB/BYTEA test.
>

I'm not sure how to usefully make such a script, since correct insertion of
BLOB data (binding with the SQL_BLOB type or similar) would work correctly
both before and after the fix.

-Dan

Re: DBD::mysql path forward

[Dan Book]

On Tue, Sep 12, 2017 at 11:54 AM, Dan Book <gri...@gmail.com> wrote:

> On Tue, Sep 12, 2017 at 11:04 AM, Patrick M. Galbraith <p...@patg.net>
> wrote:
>
>> Pali,
>>
>> Yes, I agree, we'll have to create a fork pre revert and stop accepting
>> PRs
>>
>> How might we allow people time to test the fixes to give them time? Just
>> have them use the fork, I would assume?
>>
>> Regards,
>>
>> Patrick
>>
>
> To be clear, this sounds like a branch not a fork. If your plan is to
> reinstate the mysql_enable_utf8 behavior as in 4.042 rather than adding a
> new option for this behavior, then branching from 4.042 seems reasonable to
> me; but you should be very clear if this is your intended approach, as this
> is what led to many people corrupting data as they send blobs to mysql with
> the same mysql_enable_utf8 option, and expect them to accidentally not get
> encoded.
>
> -Dan
>

And one other note, if you do reinstate the 4.042 behavior, due to the
backcompat breakage at the very least this should be a 5.0 release.

-Dan

Re: DBD::mysql path forward

[Dan Book]

On Tue, Sep 12, 2017 at 11:04 AM, Patrick M. Galbraith 
wrote:

> Pali,
>
> Yes, I agree, we'll have to create a fork pre revert and stop accepting PRs
>
> How might we allow people time to test the fixes to give them time? Just
> have them use the fork, I would assume?
>
> Regards,
>
> Patrick
>

To be clear, this sounds like a branch not a fork. If your plan is to
reinstate the mysql_enable_utf8 behavior as in 4.042 rather than adding a
new option for this behavior, then branching from 4.042 seems reasonable to
me; but you should be very clear if this is your intended approach, as this
is what led to many people corrupting data as they send blobs to mysql with
the same mysql_enable_utf8 option, and expect them to accidentally not get
encoded.

-Dan

Re: [External] Fork DBD::mysql

[Dan Book]

On Tue, Aug 29, 2017 at 3:36 AM,  wrote:

> Then users of MySQL would stay with DBD::mysql and they would not have
> fixed DBD driver. We were thinking about choosing DBD::mariadb name
> for our fork. But without dropping MySQL support, so also MySQL users
> could benefit from it.
>
> On Monday 28 August 2017 18:08:20 Daniël van Eeden wrote:
> > What about splitting it in DBD::mysql and DBD::mariadb ?
> >
> >
> > On 28 August 2017 5:55:09 p.m. p...@cpan.org wrote:
> >
> > >Hello, 2 months passed since two security problems related to DBD::mysql
> > >were reported, namely CVE-2017-10789 and CVE-2017-10788. Looking at the
> > >DBD::mysql github project page and RT bugtracker, it can be seen that
> > >nothing has happened for 2 months. Some people are asking or waiting for
> > >reported problems to be fixed... Without an answer.
> > >
> > >A long discussion was started in April by a group of people who didn't
> > >like the improvements and didn't want to see the bugs fixed in
> > >DBD::mysql driver due to the fact that they probably had old and buggy
> > >legacy code which wouldn't work after DBD::mysql starts fixing bugs.
> > >
> > >From what happened it looks like that this group of people forced
> > >DBD::mysql maintainers to revert all DBD::mysql changes (with all bug
> > >fixes including security ones). And because nothing has happened for 2
> > >months, that looks like fixing those bugs is not planned.
> > >
> > >This means just one thing:
> > >
> > >  DBD::mysql is dead right now :-(
> > >
> > >There are no new commit for 2 months, no evidence of any activity. And
> > >it looks like this is what that group of people wanted. They haven't
> > >written anything, neither any comment about open security issues, nor
> > >any comment that they forced maintainers to revert a security related
> > >bug fix. Probably they don't have to fix their own code and they can use
> > >the last released version of DBD::mysql. Perfect for them, probably a
> > >catastrophe for all others which don't want to use probably-vulnerable
> > >code.
> > >
> > >But it also means that it isn't possible to compile the last DBD::mysql
> > >with the new MySQL 8 or new MariaDB versions. More linux distributions
> > >started to pack/bundle their own DBD::mysql patches just because they
> > >need to build DBD::mysql into their repositories with upgraded MariaDB.
> > >
> > >I need to say that such situation is not acceptable for *any* future or
> > >new development in Perl which uses either MySQL or MariaDB database.
> > >
> > >DBD::mysql has couple of bugs (some are security released) and fully
> > >non-working UTF-8/Unicode support. It means that any internationalized
> > >applications which use MySQL are hard to write in Perl.
> > >
> > >Because of the current situation we in the GoodData company are thinking
> > >about forking DBD::mysql. We use DBI not only with DBD::mysql, but also
> > >with DBD::Pg. The annoying bugs in DBD::mysql still require hacks and
> > >workarounds, while similar bugs were fixed in DBD::Pg years ago. Also,
> > >inability to use the new version of MySQL or MariaDB is a problem. Same
> > >for the open security issues.
> > >
> > >I would like to ask, whether somebody else is interested in DBD::mysql
> > >fork? Either as a user or as a developer? Maintaining a fork isn't a
> > >simple task, but with supporting users and contributing developers it
> > >should be easier.
> > >
> > >And specially, are MariaDB developers interested in fixing/improving
> > >DBD::mysql fork, so Perl could work better with MariaDB servers? At
> > >least it would be able to compile against the new MariaDB C-client?
> > >
>

My reply yesterday seems to not have gone through, so I am replying again
here.

Though the reversion was made with good intentions of preserving back
compat in the short term, it was accompanied by a promise of reapplying the
many important fixes that were reverted (
https://github.com/perl5-dbi/DBD-mysql/compare/4.041...4.042 ) as well as
re-adding a correct utf8 encoding option, as it is still necessary to fix
this broken behavior one way or another. Since nothing has happened,
forking may be the only reasonable option. We are currently version pinned
to 4.042 as "upgrading" to 4.043 would be a massive regression in our
codebase.

-Dan