Re: DBD::mysql path forward

[Darren Duncan]

On 2017-11-09 10:50 PM, Alexander Hartmaier wrote:

What about DBD::MariaDB as that‘s the name of the OpenSource version these days?


No, MariaDB is simply a fork of MySQL.  Both MariaDB and MySQL are Open Source. 
Saying only MariaDB is open source is wrong.


I also disagree with using the MariaDB name as that sends the wrong message. 
Keeping the MySQL name says its for all the (API-compatible) forks while using 
the MariaDB name implies that the MariaDB fork is more special than the others.


-- Darren Duncan

Re: DBD::mysql path forward

[Darren Duncan]

Michael, why can't you accept moving forward under a new module name?  Why does 
it have to be under the old name?  When people purposefully want to upgrade they 
purposefully choose the new module name in order to do so.  What is the actual 
problem in that? -- Darren Duncan


On 2017-11-09 10:59 PM, Michiel Beijen wrote:

On Fri, Nov 10, 2017 at 7:16 AM, Darren Duncan  wrote:

I agree with everything Dan said here.  Its what I proposed, in fewer words.
Do all new development under a new name, including all of Pali's work, and
leave the current name for a product with no further effort applied to
develop it. -- Darren Duncan


This is NOT an option to me - it simply can't because the world moves
forward and because of bitrot. The 'old' version - the version that
works for most people, the current version of DBD::mysql, the one
which would then receive no more maintenance as it is no longer
compiles with the latest version of libmysqlclient and it does not
compile with libmariadb. This will only get worse in the future.

I'll stick with my earlier proposal - I'll propose to go back to the
*current* latest DBD::mysql release which does not break backcompat
for our users; add the patches that we discarded when we rolled back
one by one, such as testing on many different lib/db options, memory
leaks and so on, and make a new release so we can be on the move
again.

If possible I'd want to add back the *breaking* unicode changes that
were introduced but they should be either in a separate namespace OR
under a specific configuration option.
Currently this whole thing has cost us loosing MONTHS of progress and
then MONTHS of nothing and that is simply not good.

Patrick: let me know if you're OK with this and then let's get start again!

Re: DBD::mysql path forward

[Alexander Hartmaier]

I followed the discussion silently so far as I‘m not a user of DBD:mysql.
If the changes are breaking I‘d also think a new distribution is the way to go.
What about DBD::MariaDB as that‘s the name of the OpenSource version these days?

Best regards, Alex

> Am 10.11.2017 um 07:16 schrieb Darren Duncan :
> 
>> On 2017-11-09 8:32 AM, Dan Book wrote:
>> It seems to me like the remaining option that can make everyone "happy" is 
>> the
>> previously-suggested option of maintaining a legacy branch and doing new
>> development (reinstating 4.042) in another branch which will be released as a
>> new distribution, like DBD::mysql2, by the same maintainers. (I would not 
>> favor
>> DBD::MariaDB as a name, since this new distribution would also be the favored
>> way to connect to MySQL.) After doing so DBD::mysql can be deprecated, with
>> migration instructions added to the docs, and only receive critical security
>> fixes if anything. If done this way the community should not be fractured; 
>> the
>> current module will simply be abandoned for future development, all support 
>> and
>> maintenance will move to the new one. Patrick and Michiel, what do you think?
> 
> I agree with everything Dan said here.  Its what I proposed, in fewer words.  
> Do all new development under a new name, including all of Pali's work, and 
> leave the current name for a product with no further effort applied to 
> develop it. -- Darren Duncan

Re: DBD::mysql path forward

[Michiel Beijen]

Hi Darren, Dan,

On Fri, Nov 10, 2017 at 7:16 AM, Darren Duncan  wrote:
> On 2017-11-09 8:32 AM, Dan Book wrote:
>>
>> It seems to me like the remaining option that can make everyone "happy" is
>> the
>> previously-suggested option of maintaining a legacy branch and doing new
>> development (reinstating 4.042) in another branch which will be released
>> as a
>> new distribution, like DBD::mysql2, by the same maintainers. (I would not
>> favor
>> DBD::MariaDB as a name, since this new distribution would also be the
>> favored
>> way to connect to MySQL.) After doing so DBD::mysql can be deprecated,
>> with
>> migration instructions added to the docs, and only receive critical
>> security
>> fixes if anything. If done this way the community should not be fractured;
>> the
>> current module will simply be abandoned for future development, all
>> support and
>> maintenance will move to the new one. Patrick and Michiel, what do you
>> think?
>
> I agree with everything Dan said here.  Its what I proposed, in fewer words.
> Do all new development under a new name, including all of Pali's work, and
> leave the current name for a product with no further effort applied to
> develop it. -- Darren Duncan

This is NOT an option to me - it simply can't because the world moves
forward and because of bitrot. The 'old' version - the version that
works for most people, the current version of DBD::mysql, the one
which would then receive no more maintenance as it is no longer
compiles with the latest version of libmysqlclient and it does not
compile with libmariadb. This will only get worse in the future.

I'll stick with my earlier proposal - I'll propose to go back to the
*current* latest DBD::mysql release which does not break backcompat
for our users; add the patches that we discarded when we rolled back
one by one, such as testing on many different lib/db options, memory
leaks and so on, and make a new release so we can be on the move
again.

If possible I'd want to add back the *breaking* unicode changes that
were introduced but they should be either in a separate namespace OR
under a specific configuration option.
Currently this whole thing has cost us loosing MONTHS of progress and
then MONTHS of nothing and that is simply not good.

Patrick: let me know if you're OK with this and then let's get start again!

--
Michiel

Re: DBD::mysql path forward

[Darren Duncan]

On 2017-11-09 8:32 AM, Dan Book wrote:

It seems to me like the remaining option that can make everyone "happy" is the
previously-suggested option of maintaining a legacy branch and doing new
development (reinstating 4.042) in another branch which will be released as a
new distribution, like DBD::mysql2, by the same maintainers. (I would not favor
DBD::MariaDB as a name, since this new distribution would also be the favored
way to connect to MySQL.) After doing so DBD::mysql can be deprecated, with
migration instructions added to the docs, and only receive critical security
fixes if anything. If done this way the community should not be fractured; the
current module will simply be abandoned for future development, all support and
maintenance will move to the new one. Patrick and Michiel, what do you think?


I agree with everything Dan said here.  Its what I proposed, in fewer words.  Do 
all new development under a new name, including all of Pali's work, and leave 
the current name for a product with no further effort applied to develop it. -- 
Darren Duncan

Re: DBD::mysql path forward

[Darren Duncan]

Pali, there's a very simple solution to what you said.  The old DBD::mysql does 
not get further maintenance at all.  It is simply frozen at the 4.041/3 state 
forever.  This serves the primary reason for that to exist, which is that people 
whose package managers automatically upgrade to the highest number of a 
namespace don't introduce new corruption due to a higher version changing 
behavior being relied on.  All development can be under the new namespace full 
stop, assuming what you say is true that no one would want to backport anything 
to the otherwise frozen version. -- Darren Duncan


On 2017-11-09 12:54 AM, p...@cpan.org wrote:

On Tuesday 07 November 2017 13:19:23 Darren Duncan wrote:

A maintenance branch would exist starting from the 4.041/3 release on which
further stable releases named DBD::mysql 4.x would be made, and the primary
goal of this branch is to not break/corrupt anything that relied on 4.041
behavior. So people with legacy projects that just use DBD::mysql and made
particular assumptions on its handling of Unicode or Blobs etc won't have
corruption introduced because DBD::mysql changed its handling of those
things.


As those people or projects misuse some internals of perl we cannot
guarantee anything such that which may be broken or changed by updating
any module from cpan or external source not related to DBD::mysql.

Maintaining such thing is something what I believe nobody wants. For
sure I'm not.

As already stated more times, if there is some code which depends on
some internals, it should stay conserved in tested version as it can
break by updating anything (also unrelated). There is no other option
how to achieve that code stay working and does not corrupt something
else.


People without knowledge of CPAN or the development process will
get something that just continues to work for them and doesn't corrupt.


As explained above (and also in past) it is not possible to guarantee.


For
all intents and purposes this branch would be frozen but could have select
security or bug fixes that comply with the mandate and don't involve
changing Unicode etc.


Are you going to maintain that branch with all problems which it brings?

Or is there anybody else who want to maintain such crap?

If not, then it does not make sense to talk about this. Because nobody
expressed that is going to do such thing for 6 months which is really
large amount of time.

Re: DBD::mysql path forward

[Darren Duncan]

On 2017-11-09 12:54 AM, p...@cpan.org wrote:

On Tuesday 07 November 2017 13:19:23 Darren Duncan wrote:

The whole discussion on the mailing lists that I recall and participated in
seemed to consensus on branching DBD::sqlite in order to best satisfy the

   ~~
   DBD::mysql

I hope you are talking about DBD::mysql there...


Yes, that was a typo, I was talking exclusively about DBD::mysql here.

That being said, as an aside, DBD::SQLite (and yes I meant that) is still a 
valuable case study in that they also branched, back 14 years ago when SQLite 3 
came out, using in that case DBD::SQLite2 that bundled the SQLite version 2 and 
repurposing DBD::SQLite name, and bumping from version 0.33 to 1.0, to now 
bundle the incompatible SQLite 3.  In their case, it was a break for people that 
simply upgraded the usual CPAN manner, and the fork was for people that still 
needed to use SQLite 2 in Perl.  DBD::mysql doesn't want to follow their example.


-- Darren Duncan

Re: DBD::mysql path forward

[Dan Book]

It seems to me like the remaining option that can make everyone "happy" is
the previously-suggested option of maintaining a legacy branch and doing
new development (reinstating 4.042) in another branch which will be
released as a new distribution, like DBD::mysql2, by the same maintainers.
(I would not favor DBD::MariaDB as a name, since this new distribution
would also be the favored way to connect to MySQL.) After doing so
DBD::mysql can be deprecated, with migration instructions added to the
docs, and only receive critical security fixes if anything. If done this
way the community should not be fractured; the current module will simply
be abandoned for future development, all support and maintenance will move
to the new one. Patrick and Michiel, what do you think?

-Dan

Re: DBD::mysql path forward

[pali]

On Thursday 09 November 2017 14:26:01 Peter Rabbitson wrote:
> On 11/09/2017 01:46 PM, Noel Butler wrote:
> >On 09/11/2017 21:32, p...@cpan.org wrote:
> >
> >>
> >>>What the complaints in this thread are focused on is what the *users*
> >>>want.
> >and the users want now, or will need in the near future, to build with
> >latest, stable and recommended versions of MariaDB and MySQL.
> 
> Which is a great thing to want. I want this too. This is not what the thread
> is about.
> 
> The problem stems from Pali mashing together 3(!)* independent concerns, and
> presenting them as an "all or nothing" proposition:
>  1. Inability to build against latest libmysql libs
>  2. Several security fixes
>  3. Entirely redefining how unicode is handled throughout the stack

As I already wrote, doing active development on code of DBD::mysql is
hard or probably impossible if you want to have that old behavior.

So fixing more parts of code or adding new features would lead to
breaking that behavior or would be PITA for anybody who would need to
guarantee that it would not be broken again in future.

Moreover DBD::mysql itself (C part of code) misuse libmysql resp.
libmariadb API which leads to segfaults (or silent memory corruption)
when new mariadb is used. And this would be needed properly fixed if you
want to support mariadb.

So I guess it is better that compilation is failing. Users would not be
surprised that driver really damaged data. And not just double encoded
something to UTF-8 (due to misusing API) which has always 1:1 inverse
operation and can be fixed.

This part would not be simple to fix, there would be needed to of
changes in whole driver, find all places where driver touch mariadb C
structures... This is very good candidate which can break that old
DBD::mysql behavior of handling blobs.

Re: DBD::mysql path forward

[pali]

On Thursday 09 November 2017 13:01:19 H.Merijn Brand wrote:
> On Thu, 9 Nov 2017 12:32:00 +0100, p...@cpan.org wrote:
> 
> > > Satisfy the above - and you do get the privilege of being a maintainer
> > > of a central module with 15+ years of history.  
> > 
> > Why should I be interested in maintaining something from which users
> > want impossible things?
> 
>   "I can resist anything! (except temptation)"
> 
> I, like Peter, have backward compatibility high in my goals.
> 
> e.g. I dropped 5.005 supprt on Text::CSV_XS only in June 2012. The
> reasons to keep it working under 5.005 was that it was relatively
> little extra work. I dropped it, as it seriously blocked further
> development and support features my *user-base* asked for. Lucky me
> for being part of my own user-base, so I understand the requests.
> 
> In July 2016 I dropped 5.6.0 and 5.6.1 support. Not because it slows
> development, but because these two versions do not build anymore and
> do not support the modern toolchain. Not because the community or a
> part of the community thinks the users should upgrade to a more recent
> "stable" version of perl. Yes, I know 5.6.2 is from 2003 and 14 years
> old in 6 days and 5.8.0 is even older, but maintaining a module that
> serves almost 1000 other modules on CPAN (last time I counted, it was
> 997 direct or indirect dependencies) brings a burden of responsibility
> 
> This is how the river works: If I break this module, at least 997 other
> modules will break, and that is CPAN only. What about DarkPAN?

I fully understand backward compatibility and ability to use e.g. older
versions of perl with new modules on cpan. For example my module
Email::Address::XS is just reimplementation of the Email::Address module
in C to fix Algorithmic complexity vulnerability and has backward
compatible API. And because Email::Address worked with 5.6, I tried to
achieve also 5.6 compatibility with my Email::Address::XS module.

But DBD::mysql is different thing. Here we are facing two problems:

1) Passing perl Unicode strings is broken, even if some utf8 config
   option is enabled. This is changing data, see my identity examples
   which I sent to list.

2) Probably due to history reasons, API between DBD::mysql and perl
   applications is misused in way, that is atypical for perl itself and
   insane for pure perl applications. Moreover it is something which
   cannot be guaranteed to be "stable" as is affected also by modules
   not relevant to DBD::mysql. And is pseudo-random, but deterministic.

Fixing 1) implies breaking 2) and trying to achieve 2) leads to having
1) broken forever and leads to stopped development.

Similar/same problem as 2) had also DBD::Pg and DBD::SQLite and here
maintainers rather decided to fix Unicode and drop insanity in 2). In
previous emails I also send exact versions of DBD::Pg and DBD::SQLite
in which was behavior changed. (Modulo one problem in one specific
situation which I found in DBD::Pg.)

So now we are in situation when some DBI drivers doing one thing (from
perl POV correct) and DBD::mysql different.

If there are projects which uses both DBD::Pg and DBD::mysql, or
DBD::SQLite and DBD::mysql, then their are either broken (because
DBD::mysql differs how process non-ASCII due to 2)) or they needs
special hacks when MySQL is in use...

So what can be done now?

We have a DBD::mysql which behaves differently as others DBI drivers,
users misuse internals for BLOBs, driver has broken Unicode support
which makes it useless now when UTF-8 is mandatory for applications,
plus has pseudo-random behavior.

I already suggested, look at modules on cpan which uses DBD::mysql and
ask authors/maintainers which behavior they expects. Are they really
aware how non-ASCII data are handled in DBD::mysql and that it is
differently as e.g. DBD::SQLite?

We can also send patches to those modules to fix them. I already sent
documentation update for DBD::mysql which describe how to achieve
correct handling of the BLOB data in both old and Unicode-fixed version
of the DBD::mysql. I already said that I can help with fixing code, but
nobody was interested.

> If you do not want that burden, I'd strongly advice you *NOT* to take
> on the job of (co)maintaining DBD::mysql. Period.

I already expressed that I do not want to be maintainer of DBD::mysql.

IMO problems with 2) are not possible to maintain.

> I *see* your wish to improve it (for whatever value of improvement).
> This was exactly the reason I took maint on Text::CSV_XS: I wanted,
> maybe even required, new features. The original author/maintainer was
> not interested and/or motivated enough to do it and he passed the
> module on to me. By that time I HAD NO IDEA what the impact of doing so
> had. NONE! *I* wanted new feature and improvements, and he agreed.
> 
> It took me years to fully understand the implications of my actions.
> 
> So, currently I test a release on 7+ architectures and over 150
> versions of perl before I release and 

Re: DBD::mysql path forward

[Peter Rabbitson]

On 11/09/2017 01:46 PM, Noel Butler wrote:

On 09/11/2017 21:32, p...@cpan.org wrote:



What the complaints in this thread are focused on is what the *users* 
want.
and the users want now, or will need in the near future, to build with 
latest, stable and recommended versions of MariaDB and MySQL.


Which is a great thing to want. I want this too. This is not what the 
thread is about.


The problem stems from Pali mashing together 3(!)* independent concerns, 
and presenting them as an "all or nothing" proposition:

 1. Inability to build against latest libmysql libs
 2. Several security fixes
 3. Entirely redefining how unicode is handled throughout the stack

If the ( artificial ) choice is the-above-3-together-or-nothing, then 
the logical answer is "of course we rather have nothing until someone 
else comes by and fixes 1 and 2, leaving 3 alone"


If Pali is willing to *permanently* back off from changing anything 
related to 3 ( regardless how displeasing it is ) - then this becomes a 
completely different conversation.


Not even a hint of such willingness has been demonstrated. This is what 
is being discussed.


Cheers

* There are actual 4 issues: there is also intentional breakage of 
bigint handling within the driver, which already shipped to CPAN at the 
start of 2017 and still has not been backed out, despite multiple pleas, 
including at least one large Perl shop declaring that DBD::mysql is 
unusable to them as it stands right now: 
https://github.com/perl5-dbi/DBD-mysql/pull/74

Re: DBD::mysql path forward

[H.Merijn Brand]

On Thu, 9 Nov 2017 12:32:00 +0100, p...@cpan.org wrote:

> > Satisfy the above - and you do get the privilege of being a maintainer
> > of a central module with 15+ years of history.  
> 
> Why should I be interested in maintaining something from which users
> want impossible things?

  "I can resist anything! (except temptation)"

I, like Peter, have backward compatibility high in my goals.

e.g. I dropped 5.005 supprt on Text::CSV_XS only in June 2012. The
reasons to keep it working under 5.005 was that it was relatively
little extra work. I dropped it, as it seriously blocked further
development and support features my *user-base* asked for. Lucky me
for being part of my own user-base, so I understand the requests.

In July 2016 I dropped 5.6.0 and 5.6.1 support. Not because it slows
development, but because these two versions do not build anymore and
do not support the modern toolchain. Not because the community or a
part of the community thinks the users should upgrade to a more recent
"stable" version of perl. Yes, I know 5.6.2 is from 2003 and 14 years
old in 6 days and 5.8.0 is even older, but maintaining a module that
serves almost 1000 other modules on CPAN (last time I counted, it was
997 direct or indirect dependencies) brings a burden of responsibility

This is how the river works: If I break this module, at least 997 other
modules will break, and that is CPAN only. What about DarkPAN?

If you do not want that burden, I'd strongly advice you *NOT* to take
on the job of (co)maintaining DBD::mysql. Period.

I *see* your wish to improve it (for whatever value of improvement).
This was exactly the reason I took maint on Text::CSV_XS: I wanted,
maybe even required, new features. The original author/maintainer was
not interested and/or motivated enough to do it and he passed the
module on to me. By that time I HAD NO IDEA what the impact of doing so
had. NONE! *I* wanted new feature and improvements, and he agreed.

It took me years to fully understand the implications of my actions.

So, currently I test a release on 7+ architectures and over 150
versions of perl before I release and additionally I test the latest
release against all of those 997 modules if the new release causes any
of those deps to show new failures (compared to the old release). (Some
of these modules are currently un(der)maintained and won't build on
5.24+ for whatever reason, so if that breaks, it is not my fault.

Additionally I run speed-tests: I compare the last 20 releases for
typical use and compare the speed. If it drops out of the noise range,
that is a blocker to release, even if the new feature request is wanted
by several requestors or communities.

If you're not up to this challenge, don't do it. There are companies
depending on DBD::mysql, and they won't be your friend if they need to
modify millions lines of code if you find a new API fit better to the
new features thereby breaking old code. Backward compatibility is
important. More than you think.

How angry would you be if we broke DBI?

-- 
H.Merijn Brand  http://tux.nl   Perl Monger  http://amsterdam.pm.org/
using perl5.00307 .. 5.27   porting perl5 on HP-UX, AIX, and openSUSE
http://mirrors.develooper.com/hpux/http://www.test-smoke.org/
http://qa.perl.org   http://www.goldmark.org/jeff/stupid-disclaimers/


pgp9urdKfiWHI.pgp
Description: OpenPGP digital signature

Re: DBD::mysql path forward

[pali]

On Thursday 09 November 2017 10:32:57 Peter Rabbitson wrote:
> On 11/09/2017 09:54 AM, p...@cpan.org wrote:
> >
> >As those people or projects misuse some internals of perl we cannot
> >guarantee anything such that which may be broken or changed by updating
> >any module from cpan or external source not related to DBD::mysql.
> >
> 
> Pali. This "argument" applies to a large portion of CPAN. All JSON variants,
> various web frameworks, even things as mundane as math libraries:
> https://rt.cpan.org/Public/Bug/Display.html?id=123268

Yes, it applies and people should be aware of it...

> >As already stated more times, if there is some code which depends on
> >some internals, it should stay conserved in tested version as it can
> >break by updating anything (also unrelated). There is no other option
> >how to achieve that code stay working and does not corrupt something
> >else.
> 
> Pali, this is *not* how CPAN works. Nor is this even how /usr/bin/perl
> works.

It does not matter if cpan works like this or not. More important is
that in any world (does not have to be perl) above statement is truth.
Once you are starting depending on some internals, then you must be
aware of it that internals may change at anytime if you update one
component in ecosystem.

> >Maintaining such thing is something what I believe nobody wants. For
> >sure I'm not.
> >
> 
> I will state several very obvious ( to me ) things, apologies if they seem
> condescending: I simply do not know whether the participants on this thread
> are talking about the same thing.
> 
> Pali's entire premise is flawed. In the grand scheme of things it makes
> absolutely no difference what "maintainers want". Zero.
> 
> What the complaints in this thread are focused on is what the *users* want.

As already wrote, if users want or need to depend on internals which are
subject to change or which can be changed by something irrelevant to
DBD::mysql, maintainers nor any other developers of DBD::mysql cannot do
anything with it. The only thing what users can do is not to updating
their ecosystem OR fixing their code to not use internals.

Basically if somebody is asking for something which is either not
possible or hard to achieve, then there would be few or zero people
ready to serve such thing.

So here I would ask you: Are you going to develop, maintain and ensure
these users requirements for DBD::mysql? If not, are you able to find
another people who will do it?

I spent non-trivial time on looking at current code and I could say that
above requirement is really really hard to achieve when DBD::mysql would
be actively developed.

And because current DBD::mysql is not possible to compile with last
MariaDB nor with last (beta) MySQL and fixing these parts needs active
development, it means that such requirement is even more hard.

So if somebody depends on internals, then is already bound to some MySQL
or MariaDB version (last which works with DBD::mysql) and is already
locked for upgrading MySQL/MariaDB.

> And every maintainer, Pali included, are irrevocably bound by the general
> direction of wishes of the userbase.

I think last time we did most what we could...

> The wishes are roughly ( and in order of inmportance ):
> 
> - Breakage is never silent and is rectified swiftly when reported ( and is
> ideally never argued about )

Michiel announced more times changes to list, wrote blog posts, released
two (or more?) trial releases to cpan and waited for a longer time
before releasing final version. Discussion about patches were on both RT
and github open...

What else could be done there?

> - Upgrades are noneventful as a whole, and are as modular as possible ( i.e.
> a DBD upgrade does not drag in Test2 or something similarly non-relevant )

DBD::mysql is just one module, there is not possible to make it modular.

> - The overall library seems to move in a "better direction" ( more
> performance, less memory use, support for newer protocols, etc )

New versions fixed some problems with both new and older MySQL and
MariaDB versions, added more automated tests for different versions.

> Satisfy the above - and you do get the privilege of being a maintainer of a
> central module with 15+ years of history.

Why should I be interested in maintaining something from which users
want impossible things?

> If one can't - then the user-base is better off with the module remaining
> "unmaintained" ( reusing Pali's terminology ), and Pali's brave-new-world
> plan ending up in a different namespace. DBD::MariaDB is an idea.
> 
> Cheers

Re: DBD::mysql path forward

[Peter Rabbitson]

On 11/09/2017 09:54 AM, p...@cpan.org wrote:


As those people or projects misuse some internals of perl we cannot
guarantee anything such that which may be broken or changed by updating
any module from cpan or external source not related to DBD::mysql.



Pali. This "argument" applies to a large portion of CPAN. All JSON 
variants, various web frameworks, even things as mundane as math 
libraries: https://rt.cpan.org/Public/Bug/Display.html?id=123268



As already stated more times, if there is some code which depends on
some internals, it should stay conserved in tested version as it can
break by updating anything (also unrelated). There is no other option
how to achieve that code stay working and does not corrupt something
else.


Pali, this is *not* how CPAN works. Nor is this even how /usr/bin/perl 
works.



Maintaining such thing is something what I believe nobody wants. For
sure I'm not.



I will state several very obvious ( to me ) things, apologies if they 
seem condescending: I simply do not know whether the participants on 
this thread are talking about the same thing.


Pali's entire premise is flawed. In the grand scheme of things it makes 
absolutely no difference what "maintainers want". Zero.


What the complaints in this thread are focused on is what the *users* 
want. And every maintainer, Pali included, are irrevocably bound by the 
general direction of wishes of the userbase.


The wishes are roughly ( and in order of inmportance ):

- Breakage is never silent and is rectified swiftly when reported ( and 
is ideally never argued about )


- Upgrades are noneventful as a whole, and are as modular as possible ( 
i.e. a DBD upgrade does not drag in Test2 or something similarly 
non-relevant )


- The overall library seems to move in a "better direction" ( more 
performance, less memory use, support for newer protocols, etc )



Satisfy the above - and you do get the privilege of being a maintainer 
of a central module with 15+ years of history.


If one can't - then the user-base is better off with the module 
remaining "unmaintained" ( reusing Pali's terminology ), and Pali's 
brave-new-world plan ending up in a different namespace. DBD::MariaDB is 
an idea.


Cheers

Re: DBD::mysql path forward

[pali]

Hi! I'm responding below.

On Tuesday 07 November 2017 13:19:23 Darren Duncan wrote:
> Patrick and Pali, each of you please respond to the lists to confirm that
> what I say below is what you also understand is the primary plan, and if
> not, then say why not; in prior discussion I recall you agreed with it.
> 
> Assuming they agree, the concerns of Night Light and those he is speaking
> for should be handily satisfied.
> 
> The whole discussion on the mailing lists that I recall and participated in
> seemed to consensus on branching DBD::sqlite in order to best satisfy the
   ~~
   DBD::mysql

I hope you are talking about DBD::mysql there...

> needs of all stakeholders.
> 
> There would be one version control with 2 active release branches.
> 
> A maintenance branch would exist starting from the 4.041/3 release on which
> further stable releases named DBD::mysql 4.x would be made, and the primary
> goal of this branch is to not break/corrupt anything that relied on 4.041
> behavior. So people with legacy projects that just use DBD::mysql and made
> particular assumptions on its handling of Unicode or Blobs etc won't have
> corruption introduced because DBD::mysql changed its handling of those
> things.

As those people or projects misuse some internals of perl we cannot
guarantee anything such that which may be broken or changed by updating
any module from cpan or external source not related to DBD::mysql.

Maintaining such thing is something what I believe nobody wants. For
sure I'm not.

As already stated more times, if there is some code which depends on
some internals, it should stay conserved in tested version as it can
break by updating anything (also unrelated). There is no other option
how to achieve that code stay working and does not corrupt something
else.

> People without knowledge of CPAN or the development process will
> get something that just continues to work for them and doesn't corrupt.

As explained above (and also in past) it is not possible to guarantee.

> For
> all intents and purposes this branch would be frozen but could have select
> security or bug fixes that comply with the mandate and don't involve
> changing Unicode etc.

Are you going to maintain that branch with all problems which it brings?

Or is there anybody else who want to maintain such crap?

If not, then it does not make sense to talk about this. Because nobody
expressed that is going to do such thing for 6 months which is really
large amount of time.

> The master branch would then have all the short term breaking changes
> including the 4.042 Unicode fixes and would have all the significant feature
> changes including compatibility with newer MySQL major versions.  All of its
> releases would be under a new namespace such as DBD::mysql2 and start at
> version 5.0 to signify that large changes happened which might possibly
> break code or cause corruption if user code doesn't specifically account for
> the differences.  Being a different name this is strictly opt-in, so the
> only ones using DBD::mysql2 are those that explicitly opted in to it, and by
> doing so they also opted in to fixing anything with their code that needed
> fixing to not corrupt data while using it.
> 
> The concept of having a single driver with toggled behavior eg a
> mysql_enable_proper_unicode flag was already rejected as unwieldy to
> implement, especially for such deep rooted changes as properly handling
> Unicode.
> 
> Note that I expect that most other higher-level projects that use MySQL
> would NOT branch like this, instead having internal logic or their own
> toggle to work with DBD::mysql vs DBD::mysql2, or a few may branch, but the
> key thing is that branching DBD::mysql does NOT necessitate doing so in the
> rest of the ecosystem.
> 
> -- Darren Duncan
> 
> On 2017-11-07 12:07 PM, Night Light wrote:
> >Proposed, but no made decisions posted afterwards.
> >
> >The last proposal is to re-commit the rejected 4.042 changes into the 4.043
> >master branch and only work on fixes that came after June.
> >
> >The git issue that regards improperly encoding of BLOBs is opened on April 
> >6th
> >(hence me sending the message to prevent a recurring cycle).
> >
> >https://github.com/perl5-dbi/DBD-mysql/issues/117
> >
> >On Tue, Nov 7, 2017 at 8:57 PM, Michiel Beijen wrote:
> >
> >To me, the only real option is to make a new option in DBD::mysql;
> >mysql_enable_proper_unicode or the like which you would knowingly set
> >in your application code, which will expose the new behaviour. I
> >understand this is difficult, but I really think it's the only way.
> >
> >If in the short term this is not feasible, it *could* be possible, in
> >my eyes, to release a DBD::mysql2 or similar that does *correct*
> >behaviour. Also in that case this is something the application
> >developer should set explicitly in his connection string.
> >
> >