Re: request to become co-maintainer of DBD::SQLite
Erik, Thank you for volunteering, and that help is appreciated. Also, I still haven't heard back from the normal maintainer, so I think I'm going to step up. However, real life has intruded (eg, paying clients want stuff now) so it will still be a few weeks before I begin release-managing DBD::SQLite. I currently estimate mid-March to get into it. -- Darren Duncan Erik Aronesty wrote: I do Perl and C and offer some help. Same here. I feel reasonably at home both in C and Perl, and I've written some simple XS code. I don't have any experience with DBI, I will also do what I can to help, if anyone wants it. Also, is there a repository anywhere? It's not documented in the source or anywhere that I can find.. I usually find it easier submit patches if there's a repository involved. - There are some obvious patches and minor tweaks involving threading that can fix support on Win32. Notably the CLONE method doesn't work on a win32 fork, so a workaround would be to store the threadid along with the handle and compare to the current thread id to see if a clone is needed. (I posted an patch that simply disables testing for cloning on win32, which would probably satisfy 60% of the win32 users at the outset.) - Solaris and Win32 both have problems where the DBD xsi file creation line isn't included in the Makefile. **I think this is a DBI issue... since the code for that is part of DBI's driver implementation, but I haven't actally traced it to the line that is failing to see why. I can do that if anyone wants me too, but my bet is that the DBI devs know about it already, or could take one look at SQLite's Makefile.PL and tell me why it fails or what a good workaround is. Mostly, I'm not too concerned with a lot of the my system locks up bugs, as my guess is they are issues that will be addressed in new versions SQLite's, and they are common to embedded databases that run on a mixed-bag of operating systems. But compatibility = usage = support, and that's my primary concern.
Re: request to become co-maintainer of DBD::SQLite
I do Perl and C and offer some help. Same here. I feel reasonably at home both in C and Perl, and I've written some simple XS code. I don't have any experience with DBI, I will also do what I can to help, if anyone wants it. Also, is there a repository anywhere? It's not documented in the source or anywhere that I can find.. I usually find it easier submit patches if there's a repository involved. - There are some obvious patches and minor tweaks involving threading that can fix support on Win32. Notably the CLONE method doesn't work on a win32 fork, so a workaround would be to store the threadid along with the handle and compare to the current thread id to see if a clone is needed. (I posted an patch that simply disables testing for cloning on win32, which would probably satisfy 60% of the win32 users at the outset.) - Solaris and Win32 both have problems where the DBD xsi file creation line isn't included in the Makefile. **I think this is a DBI issue... since the code for that is part of DBI's driver implementation, but I haven't actally traced it to the line that is failing to see why. I can do that if anyone wants me too, but my bet is that the DBI devs know about it already, or could take one look at SQLite's Makefile.PL and tell me why it fails or what a good workaround is. Mostly, I'm not too concerned with a lot of the my system locks up bugs, as my guess is they are issues that will be addressed in new versions SQLite's, and they are common to embedded databases that run on a mixed-bag of operating systems. But compatibility = usage = support, and that's my primary concern.
Re: [sqlite] request to become co-maintainer of DBD::SQLite
Dear Duncan, thanks for taking on this job! I have recently started using SQLite quite heavily from Perl scripts -- it is astonishingly efficient, even for simple queries on a 70 GB database (Google's Web 1T 5-gram database, in case someone's curious) with Perl callback functions -- so I'd be more than happy to see an up-to-date version of DBD::SQLite. Since I'm lazy enough to rely on OS-provided SQLite installations on various computers, I'm using at least three different old versions of SQLite in parallel, DBD::SQLite being the oldest of all ... (no compatibility problems at all, though, so kudos to all SQLite developers!). I have been stuck back at 3.4 for various issues. I do Perl and C and offer some help. Same here. I feel reasonably at home both in C and Perl, and I've written some simple XS code. I don't have any experience with DBI, which seems to have its own method of compiling C extensions for DBD modules (from a quick look at the DBD::SQLite sources). Just let us know how/whether we can help you! Best regards, Stefan Evert [ stefan.ev...@uos.de | http://purl.org/stefan.evert ]
Re: [sqlite] request to become co-maintainer of DBD::SQLite
Hi, I would like to highlight the fact the in large corporations, bumping DBI to new version is a major issue, as the module serve as a foundation for hundreds of applications, which must be retested on every change. As a result, large companies will bump DBI version every few years. Also, large companies usually prefer to use vendor provided software. Red Hat 4 is bundled with DBI 1.40, and Red Hat 5 is bundled with 1.52. While this may not be the latest and greatest, this is the reality for many development projects. My 2 cents - If possible, DBD drivers should be compatible with older version as long as practically possible. This will make newer SQLite versions viable option for most projects. Yair -Original Message- From: Darren Duncan [mailto:dar...@darrenduncan.net] Sent: Wednesday, January 14, 2009 9:19 PM To: General Discussion of SQLite Database; DBI Dev Subject: Re: [sqlite] request to become co-maintainer of DBD::SQLite These are replies to posts on the sqlite-users list. However, if there is going to be ongoing discussion I prefer it happen on the dbi-dev list. Not that sqlite-users isn't very on topic itself, dbi-dev just seems *more* on topic, I think. Clark Christensen wrote: One of my first code changes will be to require DBI 1.607+ The current DBD-SQLite works fine under older versions of DBI. So unless there's a compelling reason to do it, I would prefer you not make what seems like an arbitrary requirement. I have 2 answers to that: 1. Sure, I can avoid changing the enforced dependency requirements for now, leaving them as Matt left them. However, I will officially deprecate support for the older versions and won't test on them. If something works with the newer dependencies but not the older ones, it will be up to those using or supporting the older dependencies to supply fixes. 2. On one hand I could say, why not update your DBI when you're updating DBD::SQLite, since even the DBI added lots of fixes one should have. On the other hand, I can understand the reality that you may have other legacy modules like drivers for other old databases that might break with a DBI update. I say might, since on the other hand they might not break. Still, I'll just go the deprecation angle for now. Otherwise, it sounds like a good start. Matt must be really busy with other work. I'll be happy to contribute where I can, but no C-fu here, either :-( Thank you. Ribeiro, Glauber wrote: My only suggestion at the moment, please use the amalgamation instead of individual files. This makes it much easier to upgrade when SQLite releases a new version. Okay. Jim Dodgen wrote: I'm for the amalgamation too. the rest of you ideas are great also. excelent idea to use Audrey Tangs nameing convention. I have been stuck back at 3.4 for various issues. I do Perl and C and offer some help. Okay and thank you. -- Darren Duncan
Re: [sqlite] request to become co-maintainer of DBD::SQLite
I am not sure agree. Companies that don't upgrade DBI releases are unlikely to upgrade DBD drivers more frequently; and they're always free to use older DBD releases. We don't want to hold developers hostage to the tendency of a few companies to be slow in upgrades. At my workplace, a large corporation, we make multiple DBI and DBD::xxx releases available, and applications can choose their own versions. It'd be unfortunate if useful new DBI features would not be used by current DBD::xxx releases. That's not to say that incompatibility should be introduced just for fun. But if a DBD driver wants to use a new DBI feature, and that breaks compatibility with older DBI releases, the DBD driver author should go ahead. The Makefile.PL file for the DBD module will specify the minimal DBI release required. yair lenga wrote: Hi, I would like to highlight the fact the in large corporations, bumping DBI to new version is a major issue, as the module serve as a foundation for hundreds of applications, which must be retested on every change. As a result, large companies will bump DBI version every few years. Also, large companies usually prefer to use vendor provided software. Red Hat 4 is bundled with DBI 1.40, and Red Hat 5 is bundled with 1.52. While this may not be the latest and greatest, this is the reality for many development projects. My 2 cents - If possible, DBD drivers should be compatible with older version as long as practically possible. This will make newer SQLite versions viable option for most projects. Yair -Original Message- From: Darren Duncan [mailto:dar...@darrenduncan.net] Sent: Wednesday, January 14, 2009 9:19 PM To: General Discussion of SQLite Database; DBI Dev Subject: Re: [sqlite] request to become co-maintainer of DBD::SQLite These are replies to posts on the sqlite-users list. However, if there is going to be ongoing discussion I prefer it happen on the dbi-dev list. Not that sqlite-users isn't very on topic itself, dbi-dev just seems *more* on topic, I think. Clark Christensen wrote: One of my first code changes will be to require DBI 1.607+ The current DBD-SQLite works fine under older versions of DBI. So unless there's a compelling reason to do it, I would prefer you not make what seems like an arbitrary requirement. I have 2 answers to that: 1. Sure, I can avoid changing the enforced dependency requirements for now, leaving them as Matt left them. However, I will officially deprecate support for the older versions and won't test on them. If something works with the newer dependencies but not the older ones, it will be up to those using or supporting the older dependencies to supply fixes. 2. On one hand I could say, why not update your DBI when you're updating DBD::SQLite, since even the DBI added lots of fixes one should have. On the other hand, I can understand the reality that you may have other legacy modules like drivers for other old databases that might break with a DBI update. I say might, since on the other hand they might not break. Still, I'll just go the deprecation angle for now. Otherwise, it sounds like a good start. Matt must be really busy with other work. I'll be happy to contribute where I can, but no C-fu here, either :-( Thank you. Ribeiro, Glauber wrote: My only suggestion at the moment, please use the amalgamation instead of individual files. This makes it much easier to upgrade when SQLite releases a new version. Okay. Jim Dodgen wrote: I'm for the amalgamation too. the rest of you ideas are great also. excelent idea to use Audrey Tangs nameing convention. I have been stuck back at 3.4 for various issues. I do Perl and C and offer some help. Okay and thank you. -- Darren Duncan
Re: [sqlite] request to become co-maintainer of DBD::SQLite
These are replies to posts on the sqlite-users list. However, if there is going to be ongoing discussion I prefer it happen on the dbi-dev list. Not that sqlite-users isn't very on topic itself, dbi-dev just seems *more* on topic, I think. Clark Christensen wrote: One of my first code changes will be to require DBI 1.607+ The current DBD-SQLite works fine under older versions of DBI. So unless there's a compelling reason to do it, I would prefer you not make what seems like an arbitrary requirement. I have 2 answers to that: 1. Sure, I can avoid changing the enforced dependency requirements for now, leaving them as Matt left them. However, I will officially deprecate support for the older versions and won't test on them. If something works with the newer dependencies but not the older ones, it will be up to those using or supporting the older dependencies to supply fixes. 2. On one hand I could say, why not update your DBI when you're updating DBD::SQLite, since even the DBI added lots of fixes one should have. On the other hand, I can understand the reality that you may have other legacy modules like drivers for other old databases that might break with a DBI update. I say might, since on the other hand they might not break. Still, I'll just go the deprecation angle for now. Otherwise, it sounds like a good start. Matt must be really busy with other work. I'll be happy to contribute where I can, but no C-fu here, either :-( Thank you. Ribeiro, Glauber wrote: My only suggestion at the moment, please use the amalgamation instead of individual files. This makes it much easier to upgrade when SQLite releases a new version. Okay. Jim Dodgen wrote: I'm for the amalgamation too. the rest of you ideas are great also. excelent idea to use Audrey Tangs nameing convention. I have been stuck back at 3.4 for various issues. I do Perl and C and offer some help. Okay and thank you. -- Darren Duncan
Re: [Dbix-class] request to become co-maintainer of DBD::SQLite
This is a reply to a post on the dbix-class list. However, if there is going to be ongoing discussion I prefer it happen on the dbi-dev list, since dbi-dev just seems the most on topic, I think. Goro Fuji wrote: Just now I am concidering to maintain DBD::SQLite, and I have ideas about that: (1) to separate SQLite from DBD::SQLite (not to be bundled), which will also separate ploblems of SQLite from those of DBD::SQLite. (2) to make a new module Alien::SQLite (see other Alien::* modules), which will allow different versions of SQLite. (3) to allow use DBD::SQLite 1.15 -sqlite_version = q(3.6.8) for testing. What do you think of it? That sounds like it could be a good idea on paper. I'm not sure I know yet the degree of changes it would involve. By default I think I'll start off with the fewest changes, also now most likely using Audrey Tang's amalgamation as the point of departure, but DBD::SQLite could also go the way you say. I think this is an idea best discussed on dbi-dev or if you want to prototype and present your idea there please do so. There are probably other issues to consider that people may bring up. Thank you. -- Darren Duncan
request to become co-maintainer of DBD::SQLite
Hello Matt Sergeant, I would like to request your permission or blessing to become an official co-maintainer of the DBD::SQLite module, which is the defacto standard binding for SQLite to Perl. (Also CC'd are some other concerned parties as FYI; my apologies if I've written too many people. But this message is initially just for response by Matt, though others can write if they feel inclined, but try to keep the recipient list smaller than I just did here. Focus any discussion to dbi-dev@perl.org and modu...@perl.org as appropriate please, the former for what work needs doing and the latter for matters of module maintainership.) P.S. Or if anyone else has the tuits and wants to make a better offer to be a co-maintainer now, please do so. I am interested in the long-term success of SQLite in combination with Perl, and in the short term I am particularly interested in using the latest SQLite 3.6.8 (which adds the extremely important feature of nested transactions) with modern versions of Perl, and I am interested that it would be easy for the large number of other DBD::SQLite users to use this combination as well. I am also concerned with there apparently being a number of significant bugs in DBD::SQLite that have been reported on the RT system, some with patches, and DBD::SQLite hasn't seen new releases in awhile to either address bugs or update the bundled SQLite. A number of people I trust are seeing that this is a serious matter to address, some in the mean-time recommending use of older DBD::SQLite versions, which is itself a problem since automatic CPAN install tools would select the newest versions, and access to newer SQLite library features is missing. Now I would of course be happiest if you had the time and motivation to bring your project up to date and address its bugs. But otherwise I would like to offer you an out, and take on this responsibility myself, either alone or with partners such as yourself or other concerned parties that want to help. If you agree, then please say the word to modu...@perl.org. My CPAN account ID is DUNCAND. To summarize, this is my intention in the short term: 1. Release a new version every time there is a SQLite core library release. 2. Make only the most minimal changes to DBD::SQLite itself, to ensure that reported bugs are fixed and that it compiles on modern systems and passes its own test suite on the same. There won't be any feature additions or architectural changes initially, except where such may be highly demanded and simple. The priorities here are stability and correctness plus easy access to all the SQLite library's native features, and minimal additional features. 3. All initial releases will have version numbers ending in _NN that mark them as developer releases, so the community can test them before they become what the CPAN tools install by default. 4. Perhaps follow what Audrey Tang started and use the official amalgamated pre-compiled source files rather than the original-original source code, so users with less capable build environments can handle it. Though in the short term this will depend on which version I can get to work with fewer problems on my own machine (Mac OS X Leopard). 5. I may use an older DBD::SQLite than the current one, such as 1.12, as an initial point of departure, if doing so makes for a more trouble-free solution. 6. I will have this in a public GIT source repository and I will regularly seek feedback, help, patches, testing, etc from the user community that have a stake in this working. 7. I am assuming until corrected that the primary discussion forum for people to discuss actual work to do and patches etc for DBD::SQLite is dbi-...@perl.org. Some caveats: 1. I have very little C-fu right now and won't be able to do much in the short term besides update the SQLite source files, and apply third-party patches to the C, and make more involved fixes to the portions written in Perl. 2. It will probably be several weeks before my first release, partly because I am busy with other tasks and I also need time for feedback and discussion. 3. All my releases should be considered experimental until further notice, after a significant body of users has put them through the ringer and considered them GA quality. 4. I *will* require third parties to submit patches to bugs in the C code in order for many relevant problems to be fixed. I may be able to fix some problems myself, but otherwise I call it a division of labour, and my part is mainly about applying patches and doing the releases; others can do a lot of the actual fix patch making. Generally speaking, the bugs that get fixed are the ones people send me patches for. 5. In general I will not ever be working with blead of the core SQLite library, only its public releases, which have a thorough test suite of their own. 6. One of my first code changes will be to require
