Re: RFC: developing DBD::SQLite
David, thanks for your quick response. David E. Wheeler wrote: You *really* need to get Matt to signoff on this, IMHO. I have tried emailing Matt several times without response already. Should I try telephoning him next? For all it looks like, Matt has abandoned the module. If someone knows better, or has been in contact recently, I'd be happy to hear. So following your response, I will eliminate a lot of the change items from my plan. In particular, I will continue to bundle the SQLite library as it has always been done, and won't download anything. I will also not add the version.pm dependency and will continue numbering releases where Matt left off, such as starting at version 1.15 or jumping up a bit and starting at 1.20_01 which should be harmless. In regards to which version to use, I will also leave that behaviour as Matt had it; his version could either use the bundled version or the system version, and I'll leave selecting which to the same mechanism Matt had it, unless the users argue for a change. I think that means the bundled is the default. So then, what I'm planning to do is this then: 1. Enter DBD::SQLite into public version control using Git, with the initial version being the last one Matt published on CPAN, and then I will merge in Audrey Tang's changes that were released as DBD::SQLite::Amalgamation; essentially there is no change but that several dozen SQLite library files are replaced with a single one. 2. Then I'll update the library to the latest one on sqlite.org, and then retest and deal with anything that breaks. 3. Then I'll look at various open RT items and apply various submitted bug fixes, again testing. And I'll accept collaborative fixes from other people such as those made against the version control. 4. Then I'll put an experimental-versioned release on CPAN. In fact, despite the larger sized releases, this probably is actually much less work for me when it comes down to it due to less complexity. I really will just make the minimal changes possible from Matt's version, just to get it to run with the current SQLite library, which will now be the single amalgamation file. Also mainly the driver would just be tested and asserted by me to work with the version of SQLite bundled with it. And if users replace the bundled amalgamation file with a different amalgamation file, generally it'll act as if the replacement was the original, so this isn't really a separate case. Now I know you said prefer system, but to start I'll just leave it the way Matt had it; mind you, if there is a general consensus to change this default, I'll honor it. And my current plans are now back to what they were earlier this year when I proposed co-maintaining the driver, a focus on minimalism, before I became more ambitious in today's first proposal. To conclude, I would be quite happy to not have this responsibility. I am doing this primarily because I want SQLite to continue to succeed in the Perl world and no one is keeping the bindings up to date anymore, despite many people asking for updates. I'm doing it because no one else is. Now if Matt comes out of the shadows and addresses the languishing driver bug reports and updates the library to the new one, and continues to do so, I'm the happiest, but so far he isn't. And my email attempts have failed at a response. So, should I try to phone Matt now? Thank you. -- Darren Duncan
Re: RFC: developing DBD::SQLite
In data 27 marzo 2009 alle ore 03:30:10, Darren Duncan dar...@darrenduncan.net ha scritto: So, out of my un-paid projects, my promise to take over release management of DBD::SQLite (from the still incommunicado previous owner) has now come to the front of my queue (now that Set::Relation 0.9.0 is out) Hi Darren, I *seem* to remember, but I might be totally wrong, that ADAMK took over maintainership for DBD::SQLite some months ago. Regarding feedback on DBD::SQLite, give me some time to read all your mail. The only thing I can say right now is that DBD::SQLite rocks for me because it's a direct-from-CPAN install with no hassle, even on Windows. -- Cosimo
Re: RFC: developing DBD::SQLite
On Thu, 26 Mar 2009 19:30:10 -0700, Darren Duncan dar...@darrenduncan.net wrote: 1. I have no problem with unbundling, but how big a deal is it to keep a pretty recent version bundled for systems that do /not/ have the library installed? In my mixed environment, neither HP-UX nor AIX have it available by default Having said that, I would not object to having to install it myself, but then the Makefile.PL process should point me to what I need to do, so the detection of available files should be close to perfect. The suggested automatic download sounds sane enough. Note that YAML-LibYAML still bundles a (broken) libyaml, but they work rather close with the libyaml people 2. Version support. If you always suggest to download the library, you will be much more confident that the test will pass, as it is likely to be the same version as you tested with. (option 1, option 3, option 2 would be my preferred sequence, as you also suggested) 3. Versions. I don't strongly object, but I see no gain in 3-part versions and version.pm dependency. Certainly for older perls it is a dependency you don't /need/. Note that I have installed version on all my perl's so I don't object to that module per sé, but having evaluated the benefits for myself, I saw no gain in switching. 4. git++ :) Any thoughts on that matter, or on anything else I raised? As you move from bundled to unbundled, I would really like to have a choice to get the old behaviour through a download. How much do you care about old(er) architectures? I mean, as you will unbundle, you have no control anymore about the portability of the files DBD-SQLite-1.14 on HP-UX 10.20: Using DBI 1.607 (for perl 5.008008 on PA-RISC2.0) cc -c -I. -I/pro/lib/perl5/site_perl/5.8.8/PA-RISC2.0/auto/DBI -Ae +DAportable +Z -z -D_HPUX_SOURCE -Wl,+vnocompatwarnings -I/pro/local/include -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 +O2 +Onolimit -DVERSION=\1.14\ -DXS_VERSION=\1.14\ +Z -I/pro/lib/perl5/5.8.8/PA-RISC2.0/CORE -DSQLITE_CORE -DSQLITE_ENABLE_FTS2 -DNDEBUG=1 -DSQLITE_PTR_SZ=4 -DHAVE_USLEEP=1 -DTHREADSAFE=0 os_unix.c cpp: os_unix.c, line 2633: error 4036: Can't open include file 'dlfcn.h'. make: *** [os_unix.o] Error 1 No, that system doesn't have it. -- H.Merijn Brand http://tux.nl Perl Monger http://amsterdam.pm.org/ using porting perl 5.6.2, 5.8.x, 5.10.x, 5.11.x on HP-UX 10.20, 11.00, 11.11, 11.23, and 11.31, SuSE 10.3, 11.0, and 11.1, AIX 5.2, and Cygwin. http://mirrors.develooper.com/hpux/ http://www.test-smoke.org/ http://qa.perl.org http://www.goldmark.org/jeff/stupid-disclaimers/
Re: RFC: developing DBD::SQLite
Cosimo Streppone wrote: In data 27 marzo 2009 alle ore 03:30:10, Darren Duncan dar...@darrenduncan.net ha scritto: So, out of my un-paid projects, my promise to take over release management of DBD::SQLite (from the still incommunicado previous owner) has now come to the front of my queue (now that Set::Relation 0.9.0 is out) Hi Darren, I *seem* to remember, but I might be totally wrong, that ADAMK took over maintainership for DBD::SQLite some months ago. Well that's news to me. And great news if it is true. It also must have been a recent development (I'll look into it) as I didn't see any CPAN releases or announcements. Regarding feedback on DBD::SQLite, give me some time to read all your mail. I first made the proposal around January 12th of this year, quickly following my spreading the news that SQLite 3.6.8 came out with support for nested transactions, and getting responses that DBD::SQLite had all sorts of unresolved bugs and whatever, and seeing for myself it was long since updated. Also Audrey Tang had released SQLite 3.6.1/2 a year ago in amalgamation form, but now neither Audrey nor Matt were doing anything, and I saw no evidence that anyone else was too. So I offered to do it. The only thing I can say right now is that DBD::SQLite rocks for me because it's a direct-from-CPAN install with no hassle, even on Windows. This is also one of the advantages of using the SQLite amalgamation version that sqlite.org provides and Audrey experimentally released. The complicated pre-compilation work is done in advance and so less capable build environments like Windows can then build SQLite without having a pricey tool. Don't know if Adam's version is the amalgamation but I'll look. Anyway, to repeat a prior reply, I've decided not to unbundle the library. -- Darren Duncan
Re: RFC: developing DBD::SQLite
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 I have tried emailing Matt several times without response already. Did you cc modu...@perl.org? What did they say? They've been helpful with me in the past in tracking module owners down. In particular, I will continue to bundle the SQLite library as it has always been done, and won't download anything. +1 I will also not add the version.pm dependency and will continue numbering releases where Matt left off, such as starting at version 1.15 or jumping up a bit and starting at 1.20_01 which should be harmless. - -1. Three part version numbers are the proper way to do things. Might as well make the change now, all at once. As far as moving things to yet another mailing list, and/or yet another IRC channel on perl.org that nobody visits, I'd much prefer that things stay here on dbi-dev, so that other driver authors can learn from your experiences, and we can contribute back in turn. If it gets high traffic and extraordinarily sqlite-specific, then, yes, use another list, but I'd lean towards keeping it here if possible. Thanks for all your work in driving this forward. - -- Greg Sabino Mullane g...@turnstep.com End Point Corporation PGP Key: 0x14964AC8 200903271007 http://biglumber.com/x/web?pk=2529DF6AB8F79407E94445B4BC9B906714964AC8 -BEGIN PGP SIGNATURE- iEYEAREDAAYFAknM3fQACgkQvJuQZxSWSshf1wCePIehoa8NF3EdGwl3Bga7R6T/ Sj0AoKWmUFDBGB5+Dly0bJWn5lDPrhB7 =NO4U -END PGP SIGNATURE-
Fwd: RFC: developing DBD::SQLite
Amalgamation what the SQLite maintainers recommend, and it's the most portable, etc. So this is the right choice. Also, given the way SQLite is maintained (backward compatibility is fairly strict, they jump through hoops in their API to make it so), bundling is fine, and is intended by the designers. I agree it can be a slight hassle to update to the current amalgamation, run tests, and then release on CPAN, but that cycle can be made pretty short. Having the *ability* to use installed libraries... say on a system where the installed libs are newer/threadsafe/being tested, whatever, is a useful feature. But, in my experience, it was only needed because the DBD::Sqlite *wasn't* being well maintained. If it had been, I would have just used the new DBD::Sqlite and never worried about it. The DBD::SQlite should use a) already installed libs if *available and compatible* b) *bundled* amalgamation, which is guaranteed to be available and compatible That's it.
Re: RFC: developing DBD::SQLite
On Fri, 27 Mar 2009, Greg Sabino Mullane wrote: I have tried emailing Matt several times without response already. Did you cc modu...@perl.org? What did they say? They've been helpful with me in the past in tracking module owners down. I very rarely read mailing list mail these days - my life got too busy for it, so if you want to contact me directly you MUST NOT CC a mailing list, because otherwise my procmail will just dump it in a folder I'll never read. Matt.
Re: RFC: developing DBD::SQLite
David E. Wheeler wrote: On Mar 27, 2009, at 2:39 AM, Darren Duncan wrote: I have tried emailing Matt several times without response already. Should I try telephoning him next? For all it looks like, Matt has abandoned the module. If someone knows better, or has been in contact recently, I'd be happy to hear. Looks like he responded to someone emailing him without CC'ing a mail list. I had also done that initially, before another post CC'ing a mail list. But anyway, it doesn't matter any more as the issue is resolved. Wow, I didn't know that you would be such a pushover. ;-) I don't use DBD::SQLite myself for any production purposes, and it has been a while since I've used it at all, so you should lean harder one people who really depend on it than on opinionated assholes like me. I'm not a pushover. It's more that I wasn't strongly opinionated on the matter in the first place and I was fishing; your response led to me realizing that a simpler plan of action was better (and less work for both me and others). And that's the end of this thread I think. -- Darren Duncan
Re: RFC: developing DBD::SQLite
On Mar 28, 2009, at 12:40 AM, Darren Duncan wrote: I'm not a pushover. It's more that I wasn't strongly opinionated on the matter in the first place and I was fishing; your response led to me realizing that a simpler plan of action was better (and less work for both me and others). (Less work)++ And that's the end of this thread I think. (Thread death)++ :-) Best, David
RFC: developing DBD::SQLite
Hello, So, out of my un-paid projects, my promise to take over release management of DBD::SQLite (from the still incommunicado previous owner) has now come to the front of my queue (now that Set::Relation 0.9.0 is out), so I'm now starting to think about it in detail and get to work over the next week or two. In that vein, the first and only major design change I intend to make right from the start is to stop bundling the SQLite library with the DBI driver and so the driver will have that library as an external dependency. While one of the selling points of DBD::SQLite versus other DBI drivers in the past was that it came with everything you need, with the advent of a single file amalgamation library being provided standard from sqlite.org, as well as the increasing availability of the SQLite library as its own shared system library install, I figure it isn't too difficult now for users to either obtain the library separately or use the one that came with their system, or the DBD::SQLite installer could automatically download it similar to how some other projects download their dependencies (Rakudo Perl 6 can download Parrot for example); so I don't think the ease of use of DBD::SQLite is diminished significantly by it no longer bundling the SQLite library. On the other side, there is a lot of benefit gained from not bundling. For one thing the size of the distribution as well as the source control is cut down significantly, since the DBI driver alone is orders of magnitude smaller. For another thing, occasional needs to update for compatibility aside, DBD::SQLite will always be up to date with the newest SQLite library; users just update the library and possibly recompile the DBD::SQLite they already have. And so DBD::SQLite won't need to be updated as often; it would just need updates to address incompatible changes to the SQLite library, or to fix bugs in or update features specific to DBD::SQLite itself. Another quasi-change is that DBD::SQLite will be designed to work specifically with the amalgamation version of the source files only, not the pre-amalgamated versions; I say quasi-change since Audrey Tang already did the work to convert DBD::SQLite to work this way, in the separate ::Amalgamation distro. Compatibility-wise, my DBD::SQLite will endeavour to work with all versions of SQLite 3.y.z, though note that only 3.4.0 for which the amalgamation file was a distinct download on sqlite.org (and 3.3.14 or so was the first that amalgamation was a make target). Or more specifically, I only plan to test with the latest SQLite source library available at the time (3.6.11 currently), as well as probably whatever version comes with Mac OS X Leopard. Supporting older versions will happen as I get advocates or testers for them. I also won't explicitly drop support for any Perl or DBI versions that the current DBD::SQLite supports, but I only intend to test it with the latest DBI and Perl 5.8.x+. A minor change is I will start out with using 3-part versions and have a dependency on version.pm 0.74, which is bundled with Perl 5.10.x and an easy install otherwise. Now a specific question for you: First assume the new DBD::SQLite can look in at least 3 places for a SQLite library to use, which are: 1. An amalgamation file that the user explicitly put in the distro directory (or that was similarly slipstreamed into a copy of the distro maybe by some OS package manager); 2. A SQLite system shared library that was installed either as part of the OS or later by a user; 3. Go and automatically fetch a copy of the latest amalgamation file from sqlite.org, similarly to how Rakudo Perl 6 can go fetch a copy of its Parrot dependency from the 'net. Now assuming that, changeable config options aside, there is an automatic default order that these alternate sources will be used by a hands-free CPAN/CPANPLUS/etc install, I ask you what options should be used and in what order? Now I think we probably can all agree that if choice #1 is satisfied (the user explicitly put a file to use in the distro dir or a packager did), then it is clear-cut to try to use that or die trying. What is less clear cut is whether the next automatic default action should be to use an already installed SQLite shared library or whether to download the latest from sqlite.org. I'm thinking that between #2 and #3, that #3 (get the latest) should always be the default choice, and that users have to explicitly override, such as by setting an environment variable, or whatever already happens with DBD::SQLite, to use the system one instead. So in fact the real question then is whether I should stay with the precedent existing with the current DBD::SQLite, which effectively is download from sqlite.org by default, or whether I should change this default. Any thoughts on that matter, or on anything else I raised? And while I suppose this is technically a fork, I'll still
Re: RFC: developing DBD::SQLite
On Mar 26, 2009, at 10:30 PM, Darren Duncan wrote: Hello, So, out of my un-paid projects, my promise to take over release management of DBD::SQLite (from the still incommunicado previous owner) has now come to the front of my queue (now that Set::Relation 0.9.0 is out), so I'm now starting to think about it in detail and get to work over the next week or two. In that vein, the first and only major design change I intend to make right from the start is to stop bundling the SQLite library with the DBI driver and so the driver will have that library as an external dependency. --1. Prefer a system-installed lib, but use the bundled lib if one cannot be found on the system. Don't make this harder for people to use. While one of the selling points of DBD::SQLite versus other DBI drivers in the past was that it came with everything you need, with the advent of a single file amalgamation library being provided standard from sqlite.org, as well as the increasing availability of the SQLite library as its own shared system library install, I figure it isn't too difficult now for users to either obtain the library separately or use the one that came with their system, or the DBD::SQLite installer could automatically download it similar to how some other projects download their dependencies (Rakudo Perl 6 can download Parrot for example); so I don't think the ease of use of DBD::SQLite is diminished significantly by it no longer bundling the SQLite library. Don't download it; a lot of times modules are installed where there is no access to the Net. And those libraries that download external dependencies never work very well (see Math::Pari). On the other side, there is a lot of benefit gained from not bundling. For one thing the size of the distribution as well as the source control is cut down significantly, since the DBI driver alone is orders of magnitude smaller. For another thing, occasional needs to update for compatibility aside, DBD::SQLite will always be up to date with the newest SQLite library; users just update the library and possibly recompile the DBD::SQLite they already have. And so DBD::SQLite won't need to be updated as often; it would just need updates to address incompatible changes to the SQLite library, or to fix bugs in or update features specific to DBD::SQLite itself. These are benefits to the developer of the module, not to the end user. I don't find them compelling. Another quasi-change is that DBD::SQLite will be designed to work specifically with the amalgamation version of the source files only, not the pre-amalgamated versions; I say quasi-change since Audrey Tang already did the work to convert DBD::SQLite to work this way, in the separate ::Amalgamation distro. Don't know anything about this. Compatibility-wise, my DBD::SQLite will endeavour to work with all versions of SQLite 3.y.z, though note that only 3.4.0 for which the amalgamation file was a distinct download on sqlite.org (and 3.3.14 or so was the first that amalgamation was a make target). Or more specifically, I only plan to test with the latest SQLite source library available at the time (3.6.11 currently), as well as probably whatever version comes with Mac OS X Leopard. Supporting older versions will happen as I get advocates or testers for them. I also won't explicitly drop support for any Perl or DBI versions that the current DBD::SQLite supports, but I only intend to test it with the latest DBI and Perl 5.8.x+. If you don't test it with other versions, how can you be sure that they're supported? I deal with this with pgTAP, BTW; I have to keep 5 separate PostgreSQL installations around to make sure it works with all of them. You could probably script it to test that it works with n versions of SQLite. That would obviously be an improvement over the current maintenance. A minor change is I will start out with using 3-part versions and have a dependency on version.pm 0.74, which is bundled with Perl 5.10.x and an easy install otherwise. Why? I see no benefit to this, and just imposes yet another inconvenience on users. Now a specific question for you: First assume the new DBD::SQLite can look in at least 3 places for a SQLite library to use, which are: 1. An amalgamation file that the user explicitly put in the distro directory (or that was similarly slipstreamed into a copy of the distro maybe by some OS package manager); 2. A SQLite system shared library that was installed either as part of the OS or later by a user; 3. Go and automatically fetch a copy of the latest amalgamation file from sqlite.org, similarly to how Rakudo Perl 6 can go fetch a copy of its Parrot dependency from the 'net. Now assuming that, changeable config options aside, there is an automatic default order that these alternate sources will be used by a hands-free CPAN/CPANPLUS/etc
