[DBD-SQLite] Re: [PATCH] Add online backup support
We're looking forward to the release of the next DBI with sqlite_ prefix support to fix the func issues ;) Kenichi ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] [PATCH] Add online backup support
Hi Toby, please write a test if you can.
As for the usage of install_method(), we need to do like this:
DBD::SQLite::db->install_method('sqlite_your_function_name');
Best,
Kenichi
On Tue, 05 May 2009 12:11:56 +1000, Toby Corkindale
wrote:
>Tim Bunce wrote:
>> Great, but please don't use the old func() interface. It's *much*
>> better for the driver to call install_method() when it loads.
>> Then applications can invoke the methods directly, without having
>> to use func(). As a bonus, RaiseError etc work with install_method()
>> but not with func().
>>
>> Either way, driver-private methods should begin with the driver's
>> prefix, 'sqlite_' in this case.
>
>Hi Tim,
>Thanks for the feedback.
>I'm not entirely sure I'm Doing It Right, but attached is a different
>patch against trunk, which uses the install_method() method of
>installing functions.
>
>It works, but produces a warning that "backup_ is not a registered
>driver prefix". However if i add "sqlite_" to the front, then I get an
>error that sqlite_backup_from_file() cannot be found.
>Although the new methods inside dbdimp.c and SQLite.xs all do have that
>prefix.
>
>Could you suggest what I have done wrong?
>
>thanks,
>Toby
>Patch to add support for SQLite's Online Backup functions.
>Signed off by Toby Corkindale
>
> SQLite.xs | 19 ++
> dbdimp.c | 56 +
> dbdimp.h |2 +
> lib/DBD/SQLite.pm | 16 +++
> 4 files changed, 93 insertions(+), 0 deletions(-)
>
>diff --git a/SQLite.xs b/SQLite.xs
>index fda00aa..8e9d0a6 100644
>--- a/SQLite.xs
>+++ b/SQLite.xs
>@@ -92,6 +92,25 @@ busy_timeout(dbh, timeout=0)
> OUTPUT:
> RETVAL
>
>+int
>+sqlite_backup_from_file(dbh, filename)
>+ SV *dbh
>+ char *filename
>+ CODE:
>+RETVAL = _sqlite_backup_from_file(aTHX_ dbh, filename);
>+ OUTPUT:
>+RETVAL
>+
>+int
>+sqlite_backup_to_file(dbh, filename)
>+ SV *dbh
>+ char *filename
>+ CODE:
>+RETVAL = _sqlite_backup_to_file(aTHX_ dbh, filename);
>+ OUTPUT:
>+RETVAL
>+
>+
> MODULE = DBD::SQLite PACKAGE = DBD::SQLite::st
>
> PROTOTYPES: DISABLE
>diff --git a/dbdimp.c b/dbdimp.c
>index 89a531e..cce3e30 100644
>--- a/dbdimp.c
>+++ b/dbdimp.c
>@@ -139,6 +139,62 @@ dbd_set_sqlite3_busy_timeout (pTHX_ SV *dbh, int timeout )
> return imp_dbh->timeout;
> }
>
>+/* Accesses the SQLite Online Backup API, and fills the currently loaded
>+ * database from the passed filename.
>+ * Usual usage of this would be when you're operating on the :memory:
>+ * special database connection and want to copy it in from a real db.
>+ */
>+int
>+_sqlite_backup_from_file(pTHX_ SV *dbh, char *filename)
>+{
>+int rc;
>+sqlite3 *pFrom;
>+sqlite3_backup *pBackup;
>+
>+D_imp_dbh(dbh);
>+
>+rc = sqlite3_open(filename, &pFrom);
>+if (rc==SQLITE_OK) {
>+
>+pBackup = sqlite3_backup_init(imp_dbh->db, "main", pFrom, "main");
>+if (pBackup) {
>+(void)sqlite3_backup_step(pBackup, -1);
>+(void)sqlite3_backup_finish(pBackup);
>+}
>+rc = sqlite3_errcode(imp_dbh->db);
>+(void)sqlite3_close(pFrom);
>+}
>+return rc;
>+}
>+
>+/* Accesses the SQLite Online Backup API, and copies the currently loaded
>+ * database into the passed filename.
>+ * Usual usage of this would be when you're operating on the :memory:
>+ * special database connection, and want to back it up to an on-disk file.
>+ */
>+int
>+_sqlite_backup_to_file(pTHX_ SV *dbh, char *filename)
>+{
>+int rc;
>+sqlite3 *pTo;
>+sqlite3_backup *pBackup;
>+
>+D_imp_dbh(dbh);
>+
>+rc = sqlite3_open(filename, &pTo);
>+if (rc==SQLITE_OK) {
>+
>+pBackup = sqlite3_backup_init(pTo, "main", imp_dbh->db, "main");
>+if (pBackup) {
>+(void)sqlite3_backup_step(pBackup, -1);
>+(void)sqlite3_backup_finish(pBackup);
>+}
>+rc = sqlite3_errcode(pTo);
>+(void)sqlite3_close(pTo);
>+}
>+return rc;
>+}
>+
> int
> sqlite_db_disconnect (SV *dbh, imp_dbh_t *imp_dbh)
> {
>diff --git a/dbdimp.h b/dbdimp.h
>index 78f7c5e..dea1bcd 100644
>--- a/dbdimp.h
>+++ b/dbdimp.h
>@@ -79,6 +79,8 @@ void sqlite3_db_progress_handler(pTHX_ SV *dbh, int
>n_opcodes, SV *handler);
> void sqlite_st_reset(pTHX_ SV *sth );
> int sqlite_bind_col( SV *sth, imp_sth_t *imp_sth, SV *col, SV *ref, IV
> sql_type, SV *attribs );
> int dbd_set_sqlite3_busy_timeout (pTHX_ SV *dbh, int timeout );
>+int _sqlite_backup_from_file(pTHX_ SV *dbh, char *filename);
>+int _sqlite_backup_to_file(pTHX_ SV *dbh, char *filename);
>
> #ifdef SvUTF8_on
>
>diff --git a/lib/DBD/SQLite.pm b/lib/DBD/SQLite.pm
>index 3e6219a..122cd14 100644
>--- a/lib/DBD/SQLite.pm
>+++ b/lib/DBD/SQLite.pm
>@@ -98,6 +98,10 @@ sub connect {
> $attr->{Warn} = 0;
> }
>
>+# Install online backup private methods:
>+DBD::SQLite::db->install_method('backup_from_file', {});
>+DBD::SQLite::db->in
Re: [DBD-SQLite] [PATCH] Add online backup support (take 2)
Hi Toby,
applied your patch with slight modification (changed function
names in dbdimp.c/h and, added a test for pre-1.608 DBI, etc).
As for the modification of your DBI, DBI 1.608 with sqlite_
prefix is out on the 5th, May (Tim++). So upgrade your DBI
and everything will be fine.
Thanks,
Kenichi
On Wed, 06 May 2009 16:09:42 +1000, Toby Corkindale
wrote:
>Please find attached my revised patch for adding online backup support
>to DBD::SQLite.
>
>This time it includes a unit test, and uses the sqlite_ prefix for
>functions, and uses the install_method() system from DBI to provide
>those functions.
>
>I note that I needed to patch my local DBI.pm to include:
>
> sqlite_ => { class => 'DBD::SQLite' }
>
>So I'm not sure what effect that will have on backward compatibility
>with current stable versions of DBI?
>
>-Toby
>Patch to add support for SQLite's Online Backup functions.
>
>I intend to improve this to use the "stepping" online backup method at some
>point, which is friendlier for concurrent database users. However the public
>API should remain the same, and I figure people might like to play with the
>system for now.
>
>Signed off by Toby Corkindale
>
> Changes |1 +
> SQLite.xs| 19
> dbdimp.c | 56 +
> dbdimp.h |2 +
> lib/DBD/SQLite.pm| 16 ++
> t/35_online_backup.t | 57 ++
> 6 files changed, 151 insertions(+), 0 deletions(-)
>
>diff --git a/Changes b/Changes
>index 4998374..2224f90 100644
>--- a/Changes
>+++ b/Changes
>@@ -9,6 +9,7 @@ Changes for Perl extension DBD-SQLite
> - Added some explanation and workarounds for a SQL that
> compares a return value of a function with a numeric bind
> value (ISHIGAKI)
>+- Added access to Online Backup functionality. (TJC)
>
> 1.25 Thu 23 Apr 2009
> - Amalgamation conversion turned out to be quicker than expected.
>diff --git a/SQLite.xs b/SQLite.xs
>index fda00aa..8e9d0a6 100644
>--- a/SQLite.xs
>+++ b/SQLite.xs
>@@ -92,6 +92,25 @@ busy_timeout(dbh, timeout=0)
> OUTPUT:
> RETVAL
>
>+int
>+sqlite_backup_from_file(dbh, filename)
>+ SV *dbh
>+ char *filename
>+ CODE:
>+RETVAL = _sqlite_backup_from_file(aTHX_ dbh, filename);
>+ OUTPUT:
>+RETVAL
>+
>+int
>+sqlite_backup_to_file(dbh, filename)
>+ SV *dbh
>+ char *filename
>+ CODE:
>+RETVAL = _sqlite_backup_to_file(aTHX_ dbh, filename);
>+ OUTPUT:
>+RETVAL
>+
>+
> MODULE = DBD::SQLite PACKAGE = DBD::SQLite::st
>
> PROTOTYPES: DISABLE
>diff --git a/dbdimp.c b/dbdimp.c
>index 89a531e..cce3e30 100644
>--- a/dbdimp.c
>+++ b/dbdimp.c
>@@ -139,6 +139,62 @@ dbd_set_sqlite3_busy_timeout (pTHX_ SV *dbh, int timeout )
> return imp_dbh->timeout;
> }
>
>+/* Accesses the SQLite Online Backup API, and fills the currently loaded
>+ * database from the passed filename.
>+ * Usual usage of this would be when you're operating on the :memory:
>+ * special database connection and want to copy it in from a real db.
>+ */
>+int
>+_sqlite_backup_from_file(pTHX_ SV *dbh, char *filename)
>+{
>+int rc;
>+sqlite3 *pFrom;
>+sqlite3_backup *pBackup;
>+
>+D_imp_dbh(dbh);
>+
>+rc = sqlite3_open(filename, &pFrom);
>+if (rc==SQLITE_OK) {
>+
>+pBackup = sqlite3_backup_init(imp_dbh->db, "main", pFrom, "main");
>+if (pBackup) {
>+(void)sqlite3_backup_step(pBackup, -1);
>+(void)sqlite3_backup_finish(pBackup);
>+}
>+rc = sqlite3_errcode(imp_dbh->db);
>+(void)sqlite3_close(pFrom);
>+}
>+return rc;
>+}
>+
>+/* Accesses the SQLite Online Backup API, and copies the currently loaded
>+ * database into the passed filename.
>+ * Usual usage of this would be when you're operating on the :memory:
>+ * special database connection, and want to back it up to an on-disk file.
>+ */
>+int
>+_sqlite_backup_to_file(pTHX_ SV *dbh, char *filename)
>+{
>+int rc;
>+sqlite3 *pTo;
>+sqlite3_backup *pBackup;
>+
>+D_imp_dbh(dbh);
>+
>+rc = sqlite3_open(filename, &pTo);
>+if (rc==SQLITE_OK) {
>+
>+pBackup = sqlite3_backup_init(pTo, "main", imp_dbh->db, "main");
>+if (pBackup) {
>+(void)sqlite3_backup_step(pBackup, -1);
>+(void)sqlite3_backup_finish(pBackup);
>+}
>+rc = sqlite3_errcode(pTo);
>+(void)sqlite3_close(pTo);
>+}
>+return rc;
>+}
>+
> int
> sqlite_db_disconnect (SV *dbh, imp_dbh_t *imp_dbh)
> {
>diff --git a/dbdimp.h b/dbdimp.h
>index 78f7c5e..dea1bcd 100644
>--- a/dbdimp.h
>+++ b/dbdimp.h
>@@ -79,6 +79,8 @@ void sqlite3_db_progress_handler(pTHX_ SV *dbh, int
>n_opcodes, SV *handler);
> void sqlite_st_reset(pTHX_ SV *sth );
> int sqlite_bind_col( SV *sth, imp_sth_t *imp_sth, SV *col, SV *ref, IV
> sql_type, SV *attribs );
> int dbd_set_sqlite3_busy_timeout (pTHX_ SV *dbh, int timeout );
>+int _sqlite_backup_from_file(pTH
Re: [DBD-SQLite] Using System SQLite Instead of Bundled Version
Hi. 1) I'm afraid Alien:: namespaces are not for the libraries like sqlite and zlib that can be easily embedded without creating extra .so/.dll files. See Compress::Raw::Zlib for example. 2) It may be nice to use a system-installed library (maybe with a selection of patches applied), but if it changes behaviors of DBD::SQLite, it'll annoy people who build their software (like O/R mappers) upon DBD::SQLite. See DBIx::Class's Makefile.PL, which tests if DBD::SQLite is not built with problematic versions of sqlite library, as it may cause unexpected errors, and the errors are most likely to be reported to them (not us, nor the sqlite team). So, it's totally ok for you to apply a patch to use a local (customized) library you have to create a binary distribution, but I'm not inclined to agree to use always a local library if available (as DBD::SQLite did before). Besides, debian is not a minor player. I think it's better to send a patch to the sqlite team, which in turn we'll include later when a new version of the library is out (than to apply a local patch that might be lost or forgotten to be applied to other libraries like DBD::SQLite). Kenichi On Thu, 21 May 2009 22:08:43 -0400, Jonathan Yu wrote: >Hi all: > >In Debian we maintain a package for the sqlite3 libraries. We also >maintain a package for DBD::SQLite, which includes a bundled version >of sqlite3. > >Judging by the Makefile.PL, there are ways to force the module to use >the system SQLite, but it has been disabled: > ># 2005/6/19, by rj...@blackperl.com ># ># Determine if we are going to use the provided SQLite code, or an already- ># installed copy. To this end, look for two command-line parameters: ># >#USE_LOCAL_SQLITE -- If non-false, force use of the installed version >#SQLITE_LOCATION -- If passed, look for headers and libs under this root ># ># In absense of either of those, expect SQLite 3.X.X libs and headers in the ># common places known to Perl or the C compiler. > ># 2009/04/02 ># But why do we need to use an older, system-installed library? ># Let's always use the bundled one. -- ISHIGAKI ># 2009/04/03 ># For the moment, while we're fixing things, this is reasonable. ># However, logic in the form "I lack knowledge, thereforce lets do ># it this way" is not a sufficiently robust decision making process. ># Let's find out the full story first, so we can make an informed ># decision to whether to do this. -- ADAMK > >>From your standpoint as DBD::SQLite developers, it makes sense - "the >system sqlite can be older than the one we're designed to work with; >how can we tell otherwise?" From a Perl developer standpoint, I think >the best place for an embedded sqlite installation is in an Alien >package. The idea with those is that you'd simply depend on >Alien::SQLite, and Alien::SQLite would install the package if >necessary. > >However, in Debian, we keep track of system packages such as sqlite3 >and can guarantee that it is the appropriate version for use with the >bindings provided in DBD::SQLite. Ideally, those command-line >parameters above would be fantastic for us, in order to use the system >SQLite instead of the local version only. > >This is because we might have patches in our sqlite3 package. If you >use the version bundled with DBD::SQLite, then obviously these patches >would be lost, which ultimately hurts our users. > >So, for what it's worth, I'd just like to say that this would be a >great install feature for us, even as currently implemented (though >disabled with the whole if (0) surrounding it). I'm curious as to why >it was disabled in the first place; is it because the code for using a >system sqlite is insufficiently robust? > >Thanks for any advice or suggestions you can offer. If there is a >better way to do this, I'd love to hear it. > >Cheers, > >Jonathan > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] releasing another DBD::SQLite version now
Hi, Darren. It's important for us, *the maintainers* to test every sqlite version to see if the latest one breaks (or fixes) DBD::SQLite or not. However, it's also important for us to give end users enough time to test their applications against the latest (stable|unstable) DBD::SQLite. It was only a month ago we released the last stable version. That's hardly enough time. As sqlite.org says, there's no compelling reason to upgrade from 3.6.13 (our bundled version) to 3.6.14, except we may benefit from minor performance improvement. 3.6.14.1 and 3.6.14.2 are only for users who upgraded to 3.6.14, so we still have no good reason to upgrade it now. Besides, I haven't finished to fix last glitches of #44871. There're still several "croak"s and I'm thinking they're simply to be removed (maybe with better argument handling) but anyway I need to add a test or two to see if my assumption is right or not, and I've been a bit busy for these weeks to do that. I asked Adam not to release too early in the commit log before. It may be ok for my part to release another dev version, but I think it's not a good time to release as sqlite is in its dev release cycle. At least we should wait sqlite 3.6.15, which probably will be out in a month or two. Kenichi On Mon, 25 May 2009 13:06:24 -0700, Darren Duncan wrote: >Adam, > >I've just updated the bundled SQLite in SVN to 3.6.14.2, which was announced 5 >hours ago. > >Would you please cut another DBD::SQLite developer release soon (or stable >release if it isn't too soon from the last one, considering that some >developers >don't want to do exhaustive retesting frequently)? The last CPAN release had >3.6.13. > >Thank you. -- Darren Duncan > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
[DBD-SQLite] Status report
Hi. As there're several active bug reports and new features,
I think it may be useful to make it clear what I'm taking or
pending now.
1) As Dami's new collation_needed implementation turned out
to be leaking badly (as noted in a comment, we should free
the struct somehow), I tentatively disabled it, looking for
a better implementation (I may also need to disable no_accent
collation test which is somehow broken now).
2) DBD::SQLite seems to be leaking while executing statements
but only when you give an explicit database name to connect
(if you use a :memory: database, the leakage seems to be gone).
I've been investigating why it happens.
3) As for the DBD-SQLite and mod_perl issue reported on RT,
I couldn't reproduce the error, but I'm guessing it's probably
due to a sqlite3_ function before sqlite_error, which might
hide a real error(*). Anyway I need a solid test for this,
and I'm afraid fixing this would require rewriting several
tests (as error messages will change in some cases). So I've
been pending this until other things are settled.
* http://www.sqlite.org/c3ref/errcode.html
4) For the improper utf8 tests issue, I'm wondering what's
the best solution. Just removing 'require utf8' line is easy
and I know the tests work fine without that line, but is
that good enough?
5) For {TYPE} attribute and SQL_IDENTIFIER_QUOTE_CHAR issues,
I'm afraid they also might break something. So I don't want to
include them in the next (dev) release that already has lots
of things to be tested.
Any help would be appreciated :)
Thanks,
Kenichi
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Status report
Hi. On Fri, 24 Jul 2009 11:24:11 +0200, "Dami Laurent (PJ)" wrote: >I can volunteer to work on this, but don't know how to detect leaks; what are >your tools ? As Jonathan said, I also use Valgrind (actually Test::Valgrind) on a debian VM. There might be something like that on Windows but I couldn't find it (or wasn't satisfied with what I found). Kenichi ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] DBD-SQLite-Amalgamation-3.6.16
DBD::SQLite::Amalgamation 3.6.16 (the one Audrey released yesterday) is "unauthorized" release, which actually is nothing but repackaged DBD::SQLite 1.26_02 with original ::Amalgamation Changes. It won't be counted, so we don't need to care. That said, we might want to convince her not to include DBD::SQLite itself, but just to depend on it as DBD::SQLite is stable enough now (though we are still trying to make it better). Kenichi On Thu, 30 Jul 2009 14:23:53 +1000, Adam Kennedy wrote: >Unless we see a release of SQLite that recommended updating, or reach >a complete resolution to the various refactoring stuff that's being >committed into the repository, or someone hits a critical bug, I >consider the current release to be both stable and worth letting sit >for a while, so there's time for all of downstream to converge. > >The release of DBD::SQLite::Amalgamation (for me) doesn't add any >weight to the need to update. > >Adam K > >2009/7/30 Darren Duncan : >> I noticed that Audrey released another DBD-SQLite-Amalgamation, after a year >> of not, that is said to be equivalent to the latest official dev release, >> citing that it would be redundant once the official release is no longer >> dev. >> >> Considering this, is it reasonable to push an official stable release of >> some kind now that has 3.6.16, or is it better to continue to wait on that >> front? >> >> -- Darren Duncan >> >> ___ >> DBD-SQLite mailing list >> DBD-SQLite@lists.scsys.co.uk >> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite >> > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] DBD-SQLite-Amalgamation-3.6.16
Ah, I must have used a wrong word ("include").
See http://search.cpan.org/~audreyt/DBD-SQLite-Amalgamation-3.6.16/
She doesn't need to do this.
Kenichi
On Thu, 30 Jul 2009 16:41:23 +1000, Adam Kennedy
wrote:
>She needs to include it.
>
>The entire point is that you install DBD::SQLite::Amalgamation and it
>overwrites DBD::SQLite, but none of your code that uses SQLite needs
>to change.
>
>That's what it's there for.
>
>Adam K
>
>2009/7/30 Kenichi Ishigaki :
>> DBD::SQLite::Amalgamation 3.6.16 (the one Audrey released
>> yesterday) is "unauthorized" release, which actually is
>> nothing but repackaged DBD::SQLite 1.26_02 with original
>> ::Amalgamation Changes. It won't be counted, so we don't
>> need to care. That said, we might want to convince her
>> not to include DBD::SQLite itself, but just to depend on it
>> as DBD::SQLite is stable enough now (though we are still
>> trying to make it better).
>>
>> Kenichi
>>
>> On Thu, 30 Jul 2009 14:23:53 +1000, Adam Kennedy
>> wrote:
>>
>>>Unless we see a release of SQLite that recommended updating, or reach
>>>a complete resolution to the various refactoring stuff that's being
>>>committed into the repository, or someone hits a critical bug, I
>>>consider the current release to be both stable and worth letting sit
>>>for a while, so there's time for all of downstream to converge.
>>>
>>>The release of DBD::SQLite::Amalgamation (for me) doesn't add any
>>>weight to the need to update.
>>>
>>>Adam K
>>>
>>>2009/7/30 Darren Duncan :
>>>> I noticed that Audrey released another DBD-SQLite-Amalgamation, after a
>>>> year
>>>> of not, that is said to be equivalent to the latest official dev release,
>>>> citing that it would be redundant once the official release is no longer
>>>> dev.
>>>>
>>>> Considering this, is it reasonable to push an official stable release of
>>>> some kind now that has 3.6.16, or is it better to continue to wait on that
>>>> front?
>>>>
>>>> -- Darren Duncan
>>>>
>>>> ___
>>>> DBD-SQLite mailing list
>>>> DBD-SQLite@lists.scsys.co.uk
>>>> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
>>>>
>>>
>>>___
>>>DBD-SQLite mailing list
>>>DBD-SQLite@lists.scsys.co.uk
>>>http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
>>
>>
>> ___
>> DBD-SQLite mailing list
>> DBD-SQLite@lists.scsys.co.uk
>> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
>>
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] collation_needed() : new implementation
Hi. I still don't have enough time to fix things (sorry) but confirmed that the new collation_needed implementation doesn't leak any more. It may be worth a new dev release. Thanks, Kenichi On Thu, 6 Aug 2009 16:34:53 +0200, "Dami Laurent (PJ)" wrote: >Hi folks, > >For info, I just committed a new implementation for "collation_needed", >that no longer uses dynamic memory allocation, and therefore should no >longer leak. The extra information needed is just an additional field in >the imp_dbh_st struct (this makes sense because at any point in time, a >dbh cannot have more than one "collation_needed" handler -- I should >have realized that in the first place!). > >To address Adam's argument about global registry, I first thought of a >bunch of methods "register_global_collation", "list_global_collations", >etc., that would hide the internal hash implementation. But the API >became heavy, and finally it seemed more perlish to expose that stuff >as a regular hash, so that clients can use all Perl idioms for hashes >--- except that this is a "write-once" hash : any attempt to modify or >delete an existing entry raises an exception. Hackers who for any >obscure reason would need to override an existing entry can always use >the 'tied' API -- but then there is really no risk of doing it >accidentally. > >The doc has been adapted in consequence. > >I hope these solutions are satisfactory to everyone. > >Greetings, Laurent Dami >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Things to do before the next production release
Hi. 1) We can rather easily see if a compiler exists or not, but I think it too much to test if the compiler has some feature or not. And, as DBI we require also requires a working compiler, (false) unknown is not so bad (why do they have DBI when they don't have a compiler? - probably they installed it via other package mananger etc, but we can't be sure). 2) The issue is reproducable (when you give a smaller value to busy_timeout(), or a larger value to sleep() ). I'll look at it further. Kenichi On Fri, 14 Aug 2009 12:49:17 +1000, Adam Kennedy wrote: >Even though there's nothing critical that requires a release of >DBD::SQLite from the sqlite.org recommendation point of view, we've >now got a fair bit of extra features, so I'd like to start thinking >about the next prod release. > >>From the CPAN Testers side, there's two issues we need to chase down and fix. > >The first is a false unknown on some platforms with bad C compiler >setups or missing C compiler setups. > >http://www.nntp.perl.org/group/perl.cpan.testers/2009/08/msg4995356.html > >I'd like to see this detected at Makefile.PL-time and converted to a NA result. > >The second is the ongoing problem with warnings in t/08_busy.t > >We need to decide if this is something we should fix (and if so >someone needs to chase down the CPAN Testers and see if you can get >access to systems, or otherwise replicate the problem) or if we should >just remove the Test::NoWarnings usage in that test script and allow >the warning to stand. > >Otherwise, the latest dev release looks fine and I think we're pretty >close to being ready to release (pending a code review on all the new >contributions by everyone) > >Adam K > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Ideas for people on a recommended namespace for DBD::SQLite enhancement packages
As there're already several applications in the SQLite:: namespace, it might be better to use (DBD::?)SQLite::Extension:: for the ones to be loaded via load_extension(), and (DBD::?)SQLite::Module:: (or whatever) for the ones that adds a customized callback etc. Kenichi On Fri, 14 Aug 2009 21:52:37 +1000, Adam Kennedy wrote: >Now that we've got a number of integration points available for >feeding code into SQLite's internals, I'm wondering if we should >consider some form of blessed or recommended namespace for people to >write these extensions in. > >Are we just going to recommend SQLite:: or does anyone have alternatives? > >Adam K > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Adding support for SQLCipher.
Hi, Dmitri,
sorry but I don't think it's a good idea to apply your patch
right now. I haven't tested it yet, but SQLCipher's README
says 1) building via 'amalgamation' isn't supported, 2) we must
define SQLITE_HAS_CODEC, 3) we need to link against a OpenSSL's
libcrypt iwth sha256 support. That means, if we want to enable
it, we also need those libraries (i.e. we also need to patch
Makefile.PL etc). And the SQLCipher's license looks like a bit
obscure, private one, which might not fit for a CPAN module.
Besides, you can freely issue PRAGMA (re)key="something" without
this patch. DBD::SQLite certainly ignores a password, but it's
derived from the DBI's API. I don't think it's a good idea to
use it in other way.
Thanks,
Kenichi
On Tue, 18 Aug 2009 07:30:33 -0700 (PDT), Dmitri Tikhonov
wrote:
>Hello,
>
>I have been working with SQLCipher [1] and I have noticed that DBD::SQLite
>does not work out of the box with it. In order to use an encrypted database,
>the first thing that should happen is to issue a pragma that specifies the key
>to decrypt pages. I modified the login function (the patch is attached) to
>use the password, if set, as the key.
>
>I was hoping this (or a modified, but equivalent) change could be made to the
>main line: it should not break existing code and will make DBD::SQLite work
>with SQLCipher.
>
> - Dmitri.
>
>1. "SQLCipher is an open source extension that provides transparent encryption
>of SQLite databases. Data pages are encrypted before being written to storage
>and decrypted on read." http://www.zetetic.net/software/sqlcipher/
>
>
>
>
>Index: dbdimp.c
>===
>--- dbdimp.c (revision 31)
>+++ dbdimp.c (revision 32)
>@@ -106,6 +106,28 @@
>
> sqlite3_busy_timeout(imp_dbh->db, SQL_TIMEOUT);
>
>+if (pass) {
>+/* If password is set, issue pragma to decrypt the database. Works
>+ * with SQLCipher.
>+ */
>+char pragma[0x100];
>+if (snprintf(pragma, sizeof(pragma), "PRAGMA key='%s'", pass) >=
>+sizeof(pragma))
>+{
>+sqlite_error(dbh, (imp_xxh_t*)imp_dbh, 1,
>+ strdup("password too long"));
>+return FALSE;
>+}
>+
>+if ((retval = sqlite3_exec(imp_dbh->db, pragma, NULL, NULL, &errmsg))
>+!= SQLITE_OK)
>+{
>+/* warn("failed to set pragma: %s\n", errmsg); */
>+sqlite_error(dbh, (imp_xxh_t*)imp_dbh, retval, errmsg);
>+return FALSE;
>+}
>+}
>+
> if ((retval = sqlite3_exec(imp_dbh->db, "PRAGMA empty_result_callbacks =
> ON",
> NULL, NULL, &errmsg))
> != SQLITE_OK)
>___
>DBD-SQLite mailing list
>DBD-SQLite@lists.scsys.co.uk
>http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Weird UTF-8 problem
Hi, Michael. I don't think of any big changes that would affect your issue (except for renaming "unicode" attribute to "sqlite_unicode"), especially if the only difference is the version of DBD::SQLite (and its library). Will you try every dev version between 1.25 and 1.26_06 to narrow down and see which version did it actually, and also see if the lost always happens with the previous versions? As for the second question, I hope so but we definitely need more tests, especially for the new and fixed features (foreign keys and other virtual table issues, various errors revealed recently, transaction and rollback, etc), and more feedback by other players including DBIC. I've been trying to convert some of the tests for the sqlite library itself with limited success, but anyway, adding some tests would help us much. Best, Kenichi On Mon, 02 Nov 2009 15:44:05 +0100, Michael Lackhoff wrote: >On 02.11.2009 14:05 Jonathan Yu wrote: > >> Have you consulted the changelog for DBD::SQLite and for upstream >> sqlite versions? If this was a fix, take a diff between the two and >> try to create a patch for it yourself. > >Well, I asked because I couldn't find anything myself. "Unicode" is >mentioned twice in the changelog but both items don't look relevant. >With regard to diffs I am very limited because I don't know any C so I >won't be able to make any sense of it (at least of the sqlite and XS >source). >I was hoping the coder who made the relevant change reads this list and >would remember. > >>> - is there a chance to make a new stable version? Then at least it is >>> easier to meet the requirement. >> I think that version 1.26 is under some testing before they want to >> cut a real "stable" release. This testing is necessary to make sure >> mass breakages don't happen as a result of the upload. Various people >> need to make sure it doesn't break other things like DBIx::Class, etc. > >That goes without saying. I don't want a stable version "just for me". >So let me rephrase my question: Do you think 1.26 has reached a maturity >that a new stable release is possible in the not too distant future? > >-Michael > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Removing the on-by-default referential integrity.
Then let's wait for another month and another sqlite release. Releasing just before this Christmas would make more sense. In the end, the current sqlite is the first version with foreign keys support. They are doing pubic tests right now, and we haven't seen, and will see the result probably in a month or so. Why do we need to rush out our stable release? As I wrote in the previous mail, we need more tests. As of this writing, we have virtually no tests for foreign keys and virtual tables they use. Besides, we have #48600 that reported several downstream distributions were revealed broken by our more strict error handling, which I haven't checked fully but it looks like they still have issues like this: http://rt.cpan.org/Public/Bug/Display.html?id=50591 Probably we should let people know that sqlite has been supporting "IF (NOT) EXISTS" for some (or a long?) time, and they can fix these issue with that clause right now, even before our next stable release. A few nights ago, DBIC people also found this issue, and they said maybe their issue can be fixed in DBIC. It's better to give people more time to test. I think removing the on-by-default bit doesn't help, especially if it's to release early. It will eventually be turned on. And most probably they know how to cope with it when it's enabled. As foreign keys have long been ignored, if they are already there, they are for other engines people are using in other places. Kenichi On Tue, 3 Nov 2009 11:03:09 +1100, Adam Kennedy wrote: >For the first production release of DBD::SQLite with foreign keys, >it's starting to make me nervous that we will enable it by default. > >As things currently stand, nobody that is using SQLite has ever seen >this feature before. They haven't had the chance to work with it at >all before we shove it down their throats. > >I think I'd like to follow SQLite itself for now and default it off. > >Thoughts? > >Adam K > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Weird UTF-8 problem (solved)
Both "sqlite_unicode" and "unicode" attributes access
the same internal handle attribute (imp_dbh->unicode),
so basically it shouldn't matter which attribute name
is used in the DBD::SQLite arena. However, it's possible
your application, or your O/R mapper maybe, does something
for you when it has its own "unicode" attribute as a connect
option. Thanks for the report.
Kenichi
On Mon, 02 Nov 2009 21:18:52 +0100, Michael Lackhoff
wrote:
>>> I will check the 1.26 dev versions.
>>
>> I just checked them all. The change is from 1.26_05 to 1.26_06. All
>> versions from 1.25 to 1.26_05 loose the UTF-8 flag, only 1.26_06 is ok.
>
>Found it. I had a connect option "sqlite_unicode => 1" plus this
>attribute setting: $dbh->{unicode} = 1;
>My thought was that this would satisfy both attribute names (or just
>because I tried everything to get it working). But in the web app the
>$dbh->{unicode} setting was ignored for whatever reason. Everything
>started working as soon as I changed the connect option to
>"unicode => 1". This was recognized by all the versions (but won't for
>future versions?)
>
>Sorry for all the hazzle
>-Michael
>
>___
>DBD-SQLite mailing list
>DBD-SQLite@lists.scsys.co.uk
>http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Removing the on-by-default referential integrity.
Our tests are hardly thorough and complete, and while I tried to write a test for foreign keys, I was hit by a weird bug that suggested the internal sqlite3 object and the DBI/DBD::SQLite handle objects were holding different statuses; the same statement works fine when issued one by one with do, and fails with consecutive executes. This may not be a showstopper, but is certainly annoying. (I haven't added the test yet, thoguh. I don't want it to be disabled again like the one I added just before 1.26_05). Anyway, I agree to comment out the pragma to turn off the default foreign keys support tentatively. But I do insist we should wait at least for two weeks, until the sqlite team release the next monthly update. Kenichi On Wed, 4 Nov 2009 14:28:12 +1100, Adam Kennedy wrote: >The fact support is still light is all the more reason to get optional >support out there in wide distribution, so more than just this mailing >list have a chance to test it thoroughly. > >At the moment, we're holding up this testing to just the people >willing to play with potentially unstable releases. > >As for why have a prod release, because of all the other fixes and >changes we've got bundled up. We finally pass our test suite >completely everywhere, so far as I can tell from CPAN Testers. I >really want that out there. > >Adam K > >2009/11/3 Kenichi Ishigaki : >> Then let's wait for another month and another sqlite release. >> Releasing just before this Christmas would make more sense. >> In the end, the current sqlite is the first version with >> foreign keys support. They are doing pubic tests right now, >> and we haven't seen, and will see the result probably in a >> month or so. Why do we need to rush out our stable release? >> >> As I wrote in the previous mail, we need more tests. As of >> this writing, we have virtually no tests for foreign keys >> and virtual tables they use. >> >> Besides, we have #48600 that reported several downstream >> distributions were revealed broken by our more strict >> error handling, which I haven't checked fully but it looks >> like they still have issues like this: >> >> http://rt.cpan.org/Public/Bug/Display.html?id=50591 >> >> Probably we should let people know that sqlite has been >> supporting "IF (NOT) EXISTS" for some (or a long?) time, >> and they can fix these issue with that clause right now, >> even before our next stable release. A few nights ago, >> DBIC people also found this issue, and they said maybe >> their issue can be fixed in DBIC. It's better to give >> people more time to test. >> >> I think removing the on-by-default bit doesn't help, >> especially if it's to release early. It will eventually >> be turned on. And most probably they know how to cope with >> it when it's enabled. As foreign keys have long been ignored, >> if they are already there, they are for other engines people >> are using in other places. >> >> >> Kenichi >> >> On Tue, 3 Nov 2009 11:03:09 +1100, Adam Kennedy >> wrote: >> >>>For the first production release of DBD::SQLite with foreign keys, >>>it's starting to make me nervous that we will enable it by default. >>> >>>As things currently stand, nobody that is using SQLite has ever seen >>>this feature before. They haven't had the chance to work with it at >>>all before we shove it down their throats. >>> >>>I think I'd like to follow SQLite itself for now and default it off. >>> >>>Thoughts? >>> >>>Adam K >>> >>>___ >>>DBD-SQLite mailing list >>>DBD-SQLite@lists.scsys.co.uk >>>http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite >> >> >> ___ >> DBD-SQLite mailing list >> DBD-SQLite@lists.scsys.co.uk >> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite >> ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Removing the on-by-default referential integrity.
OK. Agreed. Kenichi On Tue, 03 Nov 2009 21:28:59 -0800, Darren Duncan wrote: >Kenichi Ishigaki wrote: >> Our tests are hardly thorough and complete, and while I tried >> to write a test for foreign keys, I was hit by a weird bug >> that suggested the internal sqlite3 object and the DBI/DBD::SQLite >> handle objects were holding different statuses; the same statement >> works fine when issued one by one with do, and fails with consecutive >> executes. This may not be a showstopper, but is certainly annoying. >> (I haven't added the test yet, thoguh. I don't want it to be disabled >> again like the one I added just before 1.26_05). >> >> Anyway, I agree to comment out the pragma to turn off the default >> foreign keys support tentatively. But I do insist we should wait >> at least for two weeks, until the sqlite team release the next >> monthly update. > >If you mean we should wait 2 weeks and then issue the 1.27-stable as soon as >SQLite 3.6.20 comes out and include that, for the main reason that this would >include the first batch of fixes (if any) to SQLite itself following its >foreign-keys major update, then that sounds reasonable, so our testers of >foreign keys get those fixes. Especially relevant if the changes to add >foreign >key enforcement might have broken something unrelated to foreign keys. But we >should wait no longer than 3.6.20 to issue our own stable release. And our >stable would have foreign keys disabled by default. -- Darren Duncan > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] added support for FTS3
Hi, I just svn-updated and tried to compile the latest trunk but failed with a series of syntax errors. It looks like there's something more to add for older perl (I'll look into it further later; I must be leaving now). Stefan, would you send me (or to the list) the result of "prove -bvw t/rt_44891_string_looks_like_numbers.t", with your sqlite3 executable version? I haven't written in Changes yet but I was trying to fix the lingering string-looks-like-numbers issue where DBD::SQLite treated everything as text and quoted (unless you explicitly use bind_params). I've listed the known issues in t/rt_44891_string_looks_like_numbers.t. If you think of any number/text that would be apt to change and break things, please add it to the @value array, and/or prior_DBD_SQLITE_1_30_behaviors/sqlite3_bin_behaviors. Thanks, Kenichi ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] SQLite 3.7.x problem fixed, please dev-release
Hi. Sorry that I had little time recently and didn't fix the issue on sqlite numbers for 64 bit environments. I tentatively disabled it as it is rather a big change and might break other distributions' tests that expected every bind param was a quoted text for sqlite. I'll look into it again after we ship the next non-dev release. Best, Kenichi On Wed, 25 Aug 2010 11:19:59 +1000, Adam Kennedy wrote: >Will do > >On 25 Aug 2010 09:53, "Darren Duncan" wrote: >> Update ... >> >> Thanks to Niko Tyni (Debian Perl Group), a patch to fix the >incompatibility with >> SQLite 3.7.x has been handed to us via RT ( >> https://rt.cpan.org/Public/Bug/Display.html?id=60698 ). It was just 3 >lines for >> test.pm. >> >> The patch was written against 1.29 but I tested it against both 1.30_03 >and >> trunk and it fixed the test failures under 3.7.x for both. >> >> Since my earlier report of this problem, 3 more SQLite releases came out, >and >> I've added all 3 to DBD::SQLite at the time; 3.7.2 is now the latest; >upgrading >> to it is officially recommended for users of all prior releases. >> >> I also un-TODO'd 3 other tests which now pass, but didn't in 1.30_03. >> >> I also skipped-all t_44891_strings_look_like_numbers.t which has a few >> still-failing tests (some large integers are formatted in scientific >notation >> like floats) so that we can ship the 3.7.x fix ASAP. >> >> That test had been added after 1.30_03 came out, and fails with that >release too. >> >> Adam, can you please cut 1.30_04 now? (Don't forget to update the release >date >> in the Changes file.) >> >> The test suite all passes now, for me. >> >> The t_44891_strings_look_like_numbers.t will still need dealing with >somehow >> prior to a non-dev release, I suppose. >> >> Thank you. -- Darren Duncan >> >> ___ >> DBD-SQLite mailing list >> DBD-SQLite@lists.scsys.co.uk >> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] ok to add a Makefile.PL knob for enabling RTREE support
Hi. Please see the ExtUtils::MakeMaker's pod and try "perl Makefile.PL DEFINE=-DDBD_SQLITE_ENABLE_RTREE" or whatever. You'll get what you want. As for CPAN clients, you can tweak a configuration option like "makepl_arg" (permanently or tentatively). Hope this helps. Kenichi On Thu, 26 Aug 2010 09:55:00 -0700, George Hartzell wrote: > >I use SQLITE's Rtree code and as the Makefile.PL currently stands I >need to unpack it, add -DSQLITE_ENABLE_RTREE' to @CC_DEFINE, repack >the tarball and then install it by hand instead of just being able to >use cpan. > >I'd like to propose (and I'd be happy to generate a patch) something >along the lines of adding a check for an environment variable like >DBD_SQLITE_ENABLE_RTREE and if it's present/true/... then adjust >@CC_DEFINE. > >Any reason this approach won't work or is a bad idea? > >Thanks, > >g. > > >___ >DBD-SQLite mailing list >DBD-SQLite@lists.scsys.co.uk >http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] SQLite 3.7.x problem fixed, please dev-release
Fixed the Test::NoWarnings issue (hopefully). Bumped up the version, and updated Changes. Please ship the next dev one. Thanks, Kenichi On Fri, 27 Aug 2010 17:51:56 +1000, Adam Kennedy wrote: >Quite a few FAIL reports, mostly around Test::NoWarnings in fts3.t > >I'm on flaky connection, but if someone can fix it I'll drop another >dev release tonight. > >Adam K > >On Thu, Aug 26, 2010 at 5:38 AM, Darren Duncan wrote: >> Kenichi Ishigaki wrote: >>> >>> Hi. Sorry that I had little time recently and didn't fix >>> the issue on sqlite numbers for 64 bit environments. >>> I tentatively disabled it as it is rather a big change >>> and might break other distributions' tests that expected >>> every bind param was a quoted text for sqlite. I'll look >>> into it again after we ship the next non-dev release. >>> >>> Best, >>> >>> Kenichi >> >> Thank you for the timeliness of these corrections, which got in prior to >> Adam making the developer release. >> >> Under the circumstances, I recommend releasing what we have now as version >> 1.31 production within a week if there are no showstopper problems >> discovered with 1.30_04 in the meantime. >> >> In preparation for that, I'll send out a few emails now asking for people to >> test 1.30_04, which has some large changes from 1.29. >> >> -- Darren Duncan >> ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
[DBD-SQLite] source distribution?
Hi all, obra suggested me that it might be better for us to prepare a source distribution like DBD-SQLite-Devel (or Alien-SQLite, maybe) and not to install sqlite3.[ch] by default. As it's essential for extension authors to have the exact source files used to compile/install DBD::SQLite, I don't think it good to stop source files installation without providing any alternatives, but there may be a better way. There are several things we also need to consider: - Bundling source files is nice as it makes installation and maintenance much easier. - Installing source files is not always necessary for everyone, especially for those who don't need extensions like custom FTS tokenizers. - SQLite source files are fairly large (over 4 mb), and they are cumbersome when you create a binary distribution of something that uses DBD::SQLite (obra's case; he noticed this while he was packaging RT) - If we decide to prepare an SQLite source distribution, we need to ship two distributions when we release. - It would be possible to release source distribution separately, as soon as the upstream sqlite team release an update, though this might cause serious version inconsistency, and/or too many combinations of sqlite/DBD::SQLite versions, which we know was quite annoying (this is why we don't allow to use system sqlite). - No matter what we do, version inconsistency may occur when we upgrade DBD::SQLite or extensions. We can warn or force to upgrade others while installation process, but not sure which is better. There must be more to consider, but anyway, what do you think? ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] what is r13212 about? - fts3_tokenizer.h
I must have misread the commit log and misunderstood you forgot to update it, though there was no code change and no need to update it. What was necessary in fact was to set svn:eol-style to native, and I set that at rev 13271. Sorry. Kenichi 2010/11/26 Darren Duncan : > I have a question about an unrecognized change, either for ishigaki or > anyone. > > With commit r13149 I ran the update script to bring in 3.7.3, and with > commit r13212 ishigaki is saying I forgot to do something. I don't know > what this means. > > As near as I can tell, the only thing that r13212 did was change > fts3_tokenizer.h from using Unix linebreaks to Windows linebreaks, but all > the other files use Unix linebreaks, and that is a best practice for CPAN > modules, so what is special about the changed file? > > -- Darren Duncan > > === > > r13212 | ishig...@cpan.org | 2010-10-30 16:45:30 -0700 (Sat, 30 Oct 2010) | > 1 line > > DUNCAND forgot to update fts3_tokenizer.h > > r13149 | dunc...@cpan.org | 2010-10-10 17:04:56 -0700 (Sun, 10 Oct 2010) | 1 > line > > DBD::SQLite - update bundled SQLite to 3.7.3 > > > ___ > DBD-SQLite mailing list > DBD-SQLite@lists.scsys.co.uk > http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite > ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] substr w/ two parameters
Hi,
the following test script works for me on debian, centos, and windows.
Please make sure your DBD::SQLite is compiled with the bundled library.
I'm afraid that your DBD::SQLite is possibly compiled with an older version
of system sqlite.
#!/usr/bin/perl
use strict;
use warnings;
use DBI;
use Test::More tests => 4;
my $dbh = DBI->connect('dbi:SQLite::memory:');
cmp_ok $DBD::SQLite::VERSION, '>', 1.14, "DBD::SQLite $DBD::SQLite::VERSION";
cmp_ok $dbh->{sqlite_version}, 'ge', "3.6", "Compiled with
$dbh->{sqlite_version}";
$dbh->do('create table foo (text)');
$dbh->do('insert into foo values (?)', undef, "works for me?");
is $dbh->selectrow_array('select substr(text,5) from foo') => 's for me?';
is $dbh->selectrow_array('select substr(text,5,2) from foo') => 's ';
HTH,
Kenichi
2011/2/12 Jesse Erdmann :
> All,
>
> Is there a quick fix to allow substr to be called with two parameters
> instead of three? Sometime in 3.5 the official SQLite distribution
> made the third parameter optional and it looks like there is code
> within the sqlite3.c that makes queries using that form of the
> function. However, when I execute a SQL statement in my Perl code
> using either DBD::SQLite 1.31 or 1.32.1 I get an error saying prepare
> failed and that substr was being called with the wrong number of
> arguments.
>
> Using the same query in the sqlite3 command line utility executes
> appropriately. Thanks!
>
> --
> Jesse Erdmann
> Bioinformatics Analyst
> Masonic Cancer Center
> University of Minnesota
> jerdm...@umn.edu
> 612-626-3123
>
> je...@jesseerdmann.com
> Twitter: http://twitter.com/jesseerdmann
>
> ___
> DBD-SQLite mailing list
> DBD-SQLite@lists.scsys.co.uk
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
>
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] [patch] fix incorrect recipes in DBD::SQLite:: Cookbook
Applied in the trunk. Thanks.
Kenichi
On Tue, 06 Sep 2011 04:36:28 +0400, Yuriy Kaminskiy wrote:
>Hello!
>I noticed that "optimized" variance2/variance3 function in
>DBD::SQLite::Cookbook
>saves state in global variables.
>I think this is incorrect (how `SELECT variance(a), variance(b) FROM t;` is
>supposed to work? besides, they even don't reset state in new).
>Patch that fixes this error (and few others typos) attached.
>Index: DBD-SQLite-1.33/lib/DBD/SQLite/Cookbook.pod
>===
>--- DBD-SQLite-1.33.orig/lib/DBD/SQLite/Cookbook.pod 2011-09-06
>03:25:18.0 +0400
>+++ DBD-SQLite-1.33/lib/DBD/SQLite/Cookbook.pod2011-09-06
>03:55:47.0 +0400
>@@ -42,7 +42,7 @@ adapted from an example implementation i
>
> my $sigma = 0;
> foreach my $v ( @$self ) {
>- $sigma += ($x - $mu)**2;
>+ $sigma += ($v - $mu)**2;
> }
> $sigma = $sigma / ($n - 1);
>
>@@ -66,41 +66,38 @@ expense of precision:
>
> package variance2;
>
>- my $sum = 0;
>- my $count = 0;
>- my %hash;
>-
>- sub new { bless [], shift; }
>+ sub new { bless {sum => 0, count=>0, hash=> {} }, shift; }
>
> sub step {
> my ( $self, $value ) = @_;
>+ my $hash = $self->{hash};
>
> # by truncating and hashing, we can comsume many more data points
> $value = int($value); # change depending on need for precision
> # use sprintf for arbitrary fp precision
>- if (defined $hash{$value}) {
>- $hash{$value}++;
>+ if (exists $hash->{$value}) {
>+ $hash->{$value}++;
> } else {
>- $hash{$value} = 1;
>+ $hash->{$value} = 1;
> }
>- $sum += $value;
>- $count++;
>+ $self->{sum} += $value;
>+ $self->{count}++;
> }
>
> sub finalize {
> my $self = $_[0];
>
> # Variance is NULL unless there is more than one row
>- return undef unless $count > 1;
>+ return undef unless $self->{count} > 1;
>
> # calculate avg
>- my $mu = $sum / $count;
>+ my $mu = $self->{sum} / $self->{count};
>
> my $sigma = 0;
>- foreach my $h (keys %hash) {
>- $sigma += (($h - $mu)**2) * $hash{$h};
>+ while (my ($h, $v) = each %{$self->{hash}}) {
>+ $sigma += (($h - $mu)**2) * $v;
> }
>- $sigma = $sigma / ($count - 1);
>+ $sigma = $sigma / ($self->{count} - 1);
>
> return $sigma;
> }
>@@ -115,25 +112,21 @@ The function can then be used as:
>
> A third variable implementation, designed for arbitrarily large data sets:
>
>- package variance;
>-
>- my $mu = 0;
>- my $count = 0;
>- my $S = 0
>+ package variance3;
>
>- sub new { bless [], shift; }
>+ sub new { bless {mu=>0, count=>0, S=>0}, shift; }
>
> sub step {
> my ( $self, $value ) = @_;
>- $count++;
>- $delta = $value - $mu;
>- $mu = $mu + $delta/$count
>- $S = $S + $delta*($value - $mu);
>+ $self->{count}++;
>+ my $delta = $value - $self->{mu};
>+ $self->{mu} += $delta/$self->{count};
>+ $self->{S} += $delta*($value - $self->{mu});
> }
>
> sub finalize {
> my $self = $_[0];
>- return $S / ($count - 1);
>+ return $self->{S} / ($self->{count} - 1);
> }
>
> The function can then be used as:
>___
>DBD-SQLite mailing list
>DBD-SQLite@lists.scsys.co.uk
>http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Could someone explain the pod re sqlite_unicode and blobs?
Hi, Martin. I'm afraid your script has a bug:
>$s = $h->prepare(q/update test1 set b = ? where a = ?/);
>$s->execute($euro, $png);
reverse the order of params, and you'll see the difference.
Regards,
Kenichi
On Sun, 25 Sep 2011 13:12:47 +0100, "Martin J. Evans"
wrote:
>Hi,
>
> From
>http://search.cpan.org/~adamk/DBD-SQLite-1.33/lib/DBD/SQLite.pm#Database_Handle_Attributes
>
>=
>sqlite_unicode
>
> If set to a true value, DBD::SQLite will turn the UTF-8 flag on for
>all text strings coming out of the database (this feature is currently
>disabled for perl < 5.8.5). For more details on the UTF-8 flag see
>perlunicode. The default is for the UTF-8 flag to be turned off.
>
> Also note that due to some bizarreness in SQLite's type system (see
>http://www.sqlite.org/datatype3.html), if you want to retain blob-style
>behavior for some columns under $dbh->{sqlite_unicode} = 1 (say, to
>store images in the database), you have to state so explicitly using the
>3-argument form of "bind_param" in DBI when doing updates:
>
> use DBI qw(:sql_types);
> $dbh->{sqlite_unicode} = 1;
> my $sth = $dbh->prepare("INSERT INTO mytable (blobcolumn) VALUES
>(?)");
>
> # Binary_data will be stored as is.
> $sth->bind_param(1, $binary_data, SQL_BLOB);
>
> Defining the column type as BLOB in the DDL is not sufficient.
>=
>
>I don't understand this and wondered if someone could explain it better
>to me. I don't get a) the bolded "some" in "for some columns" and when
>it says "in DBI when doing updates:" it is followed with an example
>which does an insert.
>
>What I'm struggling to understand is what are the precise cases when you
>need to bind a blob as a SQL_BLOB when sqlite_unicode is on? What I've
>found if that if you do an insert or update of a png file into a blob
>field and select it back the UTF8 flag is set on but when the blob is
>inserted with TYPE => SQL_BLOB it is not - is this the problem?
>
>Here is an example:
>
>use strict;
>use warnings;
>use DBI qw(:sql_types);
>use Data::Dumper;
>use Encode;
>
>my $euro = "\x{20ac}";
>
>my $h = DBI->connect("dbi:SQLite:dbname=test.db", '', '',
> {RaiseError => 1,
> # enable the following of you won't get unicode back:
> sqlite_unicode => 1});
>eval {
> $h->do(q/drop table test1/);
>};
>$h->do(q/create table test1 (a varchar(50), b blob)/);
>
>my $s = $h->prepare(q/insert into test1 values(?, ?)/);
>
>open(my $ifh, "<:raw", "in.png");
>my $png;
>{
> local $/ = undef;
> $png = <$ifh>;
>}
># If you uncomment the following 3 lines and comment the 4th
># then blob data does not come back with UTF8 flag on
>#$s->bind_param(1, $euro);
>#$s->bind_param(2, $png, {TYPE => SQL_BLOB});
>#$s->execute;
>$s->execute($euro, $png);
>
>my $row = $h->selectrow_arrayref(q/select * from test1/);
>##print Dumper($row);
>#
>print "UTF8 flag ", Encode::is_utf8($row->[0]), "\n";
>print "UTF8 flag ", Encode::is_utf8($row->[1]), "\n";
>
>$s = $h->prepare(q/update test1 set b = ? where a = ?/);
>$s->execute($euro, $png);
>
>$row = $h->selectrow_arrayref(q/select * from test1/);
>#print Dumper($row);
>#
>print "UTF8 flag ", Encode::is_utf8($row->[0]), "\n";
>print "UTF8 flag ", Encode::is_utf8($row->[1]), "\n";
>
>open(my $ofh, ">:raw", "out.png");
>print $ofh $row->[1];
>close $ofh;
>
>which produces:
>
>UTF8 flag 1
>UTF8 flag 1
>UTF8 flag 1
>UTF8 flag 1
>
>but when you make the change (see comments) you get:
>
>UTF8 flag 1
>UTF8 flag
>UTF8 flag 1
>UTF8 flag
>
>Thanks
>
>Martin
>
>___
>DBD-SQLite mailing list
>DBD-SQLite@lists.scsys.co.uk
>http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Could someone explain the pod re sqlite_unicode and blobs?
And this bind_col() issue is a bug in DBD::SQLite.
I'll write a fix and a test for it later.
Thanks,
Kenichi
On Tue, 27 Sep 2011 22:57:34 +0100, "Martin J. Evans"
wrote:
>On 25/09/2011 13:12, Martin J. Evans wrote:
>> Hi,
>>
>> From
>> http://search.cpan.org/~adamk/DBD-SQLite-1.33/lib/DBD/SQLite.pm#Database_Handle_Attributes
>>
>> =
>> sqlite_unicode
>>
>> If set to a true value, DBD::SQLite will turn the UTF-8 flag on
>> for all text strings coming out of the database (this feature is
>> currently disabled for perl < 5.8.5). For more details on the UTF-8
>> flag see perlunicode. The default is for the UTF-8 flag to be turned off.
>>
>> Also note that due to some bizarreness in SQLite's type system
>> (see http://www.sqlite.org/datatype3.html), if you want to retain
>> blob-style behavior for some columns under $dbh->{sqlite_unicode} = 1
>> (say, to store images in the database), you have to state so
>> explicitly using the 3-argument form of "bind_param" in DBI when doing
>> updates:
>>
>> use DBI qw(:sql_types);
>> $dbh->{sqlite_unicode} = 1;
>> my $sth = $dbh->prepare("INSERT INTO mytable (blobcolumn) VALUES
>> (?)");
>>
>> # Binary_data will be stored as is.
>> $sth->bind_param(1, $binary_data, SQL_BLOB);
>>
>> Defining the column type as BLOB in the DDL is not sufficient.
>> =
>>
>> I don't understand this and wondered if someone could explain it
>> better to me. I don't get a) the bolded "some" in "for some columns"
>> and when it says "in DBI when doing updates:" it is followed with an
>> example which does an insert.
>>
>> What I'm struggling to understand is what are the precise cases when
>> you need to bind a blob as a SQL_BLOB when sqlite_unicode is on? What
>> I've found if that if you do an insert or update of a png file into a
>> blob field and select it back the UTF8 flag is set on but when the
>> blob is inserted with TYPE => SQL_BLOB it is not - is this the problem?
>>
>> Here is an example:
>>
>> use strict;
>> use warnings;
>> use DBI qw(:sql_types);
>> use Data::Dumper;
>> use Encode;
>>
>> my $euro = "\x{20ac}";
>>
>> my $h = DBI->connect("dbi:SQLite:dbname=test.db", '', '',
>> {RaiseError => 1,
>> # enable the following of you won't get unicode back:
>> sqlite_unicode => 1});
>> eval {
>> $h->do(q/drop table test1/);
>> };
>> $h->do(q/create table test1 (a varchar(50), b blob)/);
>>
>> my $s = $h->prepare(q/insert into test1 values(?, ?)/);
>>
>> open(my $ifh, "<:raw", "in.png");
>> my $png;
>> {
>> local $/ = undef;
>> $png = <$ifh>;
>> }
>> # If you uncomment the following 3 lines and comment the 4th
>> # then blob data does not come back with UTF8 flag on
>> #$s->bind_param(1, $euro);
>> #$s->bind_param(2, $png, {TYPE => SQL_BLOB});
>> #$s->execute;
>> $s->execute($euro, $png);
>>
>> my $row = $h->selectrow_arrayref(q/select * from test1/);
>> ##print Dumper($row);
>> #
>> print "UTF8 flag ", Encode::is_utf8($row->[0]), "\n";
>> print "UTF8 flag ", Encode::is_utf8($row->[1]), "\n";
>>
>> $s = $h->prepare(q/update test1 set b = ? where a = ?/);
>> $s->execute($euro, $png);
>>
>> $row = $h->selectrow_arrayref(q/select * from test1/);
>> #print Dumper($row);
>> #
>> print "UTF8 flag ", Encode::is_utf8($row->[0]), "\n";
>> print "UTF8 flag ", Encode::is_utf8($row->[1]), "\n";
>>
>> open(my $ofh, ">:raw", "out.png");
>> print $ofh $row->[1];
>> close $ofh;
>>
>> which produces:
>>
>> UTF8 flag 1
>> UTF8 flag 1
>> UTF8 flag 1
>> UTF8 flag 1
>>
>> but when you make the change (see comments) you get:
>>
>> UTF8 flag 1
>> UTF8 flag
>> UTF8 flag 1
>> UTF8 flag
>>
>> Thanks
>>
>> Martin
>>
>
>I'd appreciate some feedback as I'm attempting to collate information on
>unicode support in different DBDs - how it is implemented and what are
>the issues with it so I can present it on dbi-dev to attempt to
>standardise unicode support in DBI.
>
>Further to the above, I've been experimenting and I also looked at
>https://rt.cpan.org/Ticket/Display.html?id=19471 which suggested that
>either binding on input as a SQL_BLOB or binding on output as an
>SQL_BLOB would not set the utd8 flag. I confirm the former but when
>binding a column on a select as SQL_BLOB I get no data back at all:
>
>Before this test in.png is a valid png file and out.png does not exist.
>Output is:
>
>C:>perl sqlite4.pl
>UTF8 flag 1
>UTF8 flag
>Use of uninitialized value $col2 in print at sqlite4.pl line 65, <$ifh>
>line 1.
>
>C:>dir out.png
>27/09/2011 22:44 0 out.png
>1 File(s) 0 bytes
>0 Dir(s) 15,804,985,344 bytes free
>
>Altering the script to use the bind_col without specifying the type
>produces:
>
>C:>perl sqlite4.pl
>UTF8 flag 1
>UTF8 flag
>
>C:>dir out.png
>27/09/2011 22:4612,987 out.png
>1 File(s) 12,987 bytes
>0 Dir(s) 15,804,968,960 bytes free
>
>Also selectall_arrayref with out any bound columns
Re: [DBD-SQLite] Could someone explain the pod re sqlite_unicode and blobs?
On Wed, 28 Sep 2011 08:45:16 +0100, "Martin J. Evans"
wrote:
>On 28/09/11 01:17, Kenichi Ishigaki wrote:
>> And this bind_col() issue is a bug in DBD::SQLite.
>> I'll write a fix and a test for it later.
>>
>> Thanks,
>>
>> Kenichi
>
>Do you want an rt?
Yes, please. Thanks,
Kenichi
>Martin
>
>> On Tue, 27 Sep 2011 22:57:34 +0100, "Martin J.
>> Evans" wrote:
>>
>>> On 25/09/2011 13:12, Martin J. Evans wrote:
>>>> Hi,
>>>>
>>>> From
>>>> http://search.cpan.org/~adamk/DBD-SQLite-1.33/lib/DBD/SQLite.pm#Database_Handle_Attributes
>>>>
>>>> =
>>>> sqlite_unicode
>>>>
>>>> If set to a true value, DBD::SQLite will turn the UTF-8 flag on
>>>> for all text strings coming out of the database (this feature is
>>>> currently disabled for perl< 5.8.5). For more details on the UTF-8
>>>> flag see perlunicode. The default is for the UTF-8 flag to be turned off.
>>>>
>>>> Also note that due to some bizarreness in SQLite's type system
>>>> (see http://www.sqlite.org/datatype3.html), if you want to retain
>>>> blob-style behavior for some columns under $dbh->{sqlite_unicode} = 1
>>>> (say, to store images in the database), you have to state so
>>>> explicitly using the 3-argument form of "bind_param" in DBI when doing
>>>> updates:
>>>>
>>>>use DBI qw(:sql_types);
>>>>$dbh->{sqlite_unicode} = 1;
>>>>my $sth = $dbh->prepare("INSERT INTO mytable (blobcolumn) VALUES
>>>> (?)");
>>>>
>>>># Binary_data will be stored as is.
>>>>$sth->bind_param(1, $binary_data, SQL_BLOB);
>>>>
>>>> Defining the column type as BLOB in the DDL is not sufficient.
>>>> =
>>>>
>>>> I don't understand this and wondered if someone could explain it
>>>> better to me. I don't get a) the bolded "some" in "for some columns"
>>>> and when it says "in DBI when doing updates:" it is followed with an
>>>> example which does an insert.
>>>>
>>>> What I'm struggling to understand is what are the precise cases when
>>>> you need to bind a blob as a SQL_BLOB when sqlite_unicode is on? What
>>>> I've found if that if you do an insert or update of a png file into a
>>>> blob field and select it back the UTF8 flag is set on but when the
>>>> blob is inserted with TYPE => SQL_BLOB it is not - is this the problem?
>>>>
>>>> Here is an example:
>>>>
>>>> use strict;
>>>> use warnings;
>>>> use DBI qw(:sql_types);
>>>> use Data::Dumper;
>>>> use Encode;
>>>>
>>>> my $euro = "\x{20ac}";
>>>>
>>>> my $h = DBI->connect("dbi:SQLite:dbname=test.db", '', '',
>>>> {RaiseError => 1,
>>>> # enable the following of you won't get unicode back:
>>>> sqlite_unicode => 1});
>>>> eval {
>>>> $h->do(q/drop table test1/);
>>>> };
>>>> $h->do(q/create table test1 (a varchar(50), b blob)/);
>>>>
>>>> my $s = $h->prepare(q/insert into test1 values(?, ?)/);
>>>>
>>>> open(my $ifh, "<:raw", "in.png");
>>>> my $png;
>>>> {
>>>> local $/ = undef;
>>>> $png =<$ifh>;
>>>> }
>>>> # If you uncomment the following 3 lines and comment the 4th
>>>> # then blob data does not come back with UTF8 flag on
>>>> #$s->bind_param(1, $euro);
>>>> #$s->bind_param(2, $png, {TYPE => SQL_BLOB});
>>>> #$s->execute;
>>>> $s->execute($euro, $png);
>>>>
>>>> my $row = $h->selectrow_arrayref(q/select * from test1/);
>>>> ##print Dumper($row);
>>>> #
>>>> print "UTF8 flag ", Encode::is_utf8($row->[0]), "\n";
>>>> print "UTF8 flag ", Encode::is_utf8($row->[1]), "\n";
>>>>
>>>> $s = $h->prepare(q/update test1 set b = ? where a = ?/);
>>>> $s->execute($euro, $png);
>>>>
>>>> $row = $h->selectrow_arrayref(q/select * from test1/);
>>>&g
Re: [DBD-SQLite] Re: DBD::SQLite / 32-bit Perl and 64-bit INTEGER PRIMARY KEY
Hi Matthias, 2011/10/6 Adam Kennedy : > Hi Matthias > > I may not be the best person to address this question to, as I'm just > the release manager and crotchety old fogie of DBD::SQLite. > > The question is best addressed to the DBD::SQLite mailing list. > > dbd-sqlite@lists.scsys.co.uk (forwarded) > > Adam K > > On 5 October 2011 03:33, Matthias Waldorf wrote: >> Dear Adam, >> >> sorry if this a stupid question, but I could not find out how to use a >> 64-bit INTEGER PRIMARY KEY with a 32-bit version of Perl. >> >> My primary key is a 64-bit number stored as a 16-byte hexadecimal string. Is >> there a way to specify an hexadecimal number as INTEGER PRIMARY KEY for an >> INSERT, UPDATE or SELECT statement? INTEGER PRIMARY KEY is something special for sqlite, and you can't insert anything sqlite doesn't consider an integer. Maybe you might want to drop "INTEGER" (or "PRIMARY KEY" if applicable) from your schema, and always treat your keys as hexadecimal strings (or decimal strings if you convert them to 64bit integer with the help of Math::BigInt). You might also want to provide some custom collation function if you need finer control (please refer to the DBD::SQLite pod). Hope this helps, Kenichi >> Thank you very much for your help! >> >> Matthias >> > > ___ > DBD-SQLite mailing list > DBD-SQLite@lists.scsys.co.uk > http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite > ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] Re: Regression test for this bug (was [PATCH v2] Don't set imp_sth->stmt to NULL on error path)
Hi Yuriy,
Thank you for the patches. This issue seems to have deeper roots,
so please give me some time before I apply them.
Best,
Kenichi
2011/10/6 Yuriy Kaminskiy :
> Yuriy Kaminskiy wrote:
>> Yuriy Kaminskiy wrote:
>>> Yuriy Kaminskiy wrote:
Hello!
I noticed strange problem on error path handling.
Not sure if this sqlite error, DBD::SQLite error, or some misunderstanding
on my
part.
Verified on DBD::SQLite-1.33 and 1.34_1 (with bundled sqlite.c) and
DBI-1.616.
my $sth=$dbh->prepare("SELECT * FROM t");
$sth->execute; # success - 0E0
my $row1 = $sth->fetch; # returns [1]
$sth->finish;
sleep 5;
# ^^ here another instance issued BEGIN EXCLUSIVE ...
$sth->execute; # ... so this returns "database is locked";
# it's legitimate/expected behavior
#
$sth->finish; # just in case (clears error)
#
sleep 5;
# ^^ here another instance already COMMIT'ed, DB is not locked anymore
#
$sth->execute; # success - 0E0
my $row2 = $sth->fetch; # returns undef
#
# WTF?
#
# If I prepare another statement (cloned from first!),
# it returns row as expected:
#
my $sth2 = $dbh->prepare($sth->{Statement});
$sth2->execute; # success
my $row3 = $sth2->fetch; # returns [1]
>>> With attached patch it works as expected. Seems passes regression test, even
>>> under valgrind (but maybe regression test should be improved, so that it
>>> won't
>>> pass it *before* this patch).
>>>
>>> This bug was introduced (or, maybe, "incompletely fixed") by svn revision
>>> 5944
>>> (author: adamk/subject: "Changed finalize to reset per RT #32100 et
>>> al"/date:
>>> 2009-04-04 07:01:40), that replaced sqlite3_finalize with sqlite3_reset, but
>>> have not removed stmt NULLification.
>>
>> v2: remove tabs, split unrelated changes to separate patch
>>
> Most code adopted from from t/08_busy.t
>
> ___
> DBD-SQLite mailing list
> DBD-SQLite@lists.scsys.co.uk
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
>
>
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] [patch] Error handling, part 3: document sqlite-specific $sth->finish interaction with transactions and workaround error in $dbh->rollback
I'd like to turn these down, at least for now, as SQLite doc clearly states as follows: The ROLLBACK will fail with an error code SQLITE_BUSY if there are any pending queries. Both read-only and read/write queries will cause a ROLLBACK to fail. A ROLLBACK must fail if there are pending read operations (unlike COMMIT which can succeed) because bad things will happen if the in-memory image of the database is changed out from under an active query. (http://sqlite.org/lang_transaction.html) And DBI doc also says When all the data has been fetched from a "SELECT" statement, the driver will automatically call "finish" for you. So you should *not* call it explicitly *except* when you know that you've not fetched all the data from a statement handle *and* the handle won't be destroyed soon. This rolling-back an unfinished query is obviously one of such exceptional cases. These patches also have side effects that probably surprise users. I think it's better for you to add $sth->finish explicitly before you call $sth->rollback for a reading query. Regards, Kenichi 2011/10/15 Yuriy Kaminskiy : > In sqlite, ROLLBACK fails with SQLITE_BUSY when there are active/unfinished > SELECT statements. > As DBI's $dbh->rollback cannot handle errors in sane way, add workaround to > reset all sth and retry when $dbh->rollback fails with SQLITE_BUSY. > > PS For whatever reasons, "Error handling, part 2" cannot be sent to list > (disappear into /dev/null without any diagnostics) :-| > Will retry in few days if it won't appear in gmane or list archives. > > ___ > DBD-SQLite mailing list > DBD-SQLite@lists.scsys.co.uk > http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite > > ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] DBI, DBD::SQLite and threads
Hi. I don't usually use the "threads" module in my projects, but...
1) DBD::SQLite has several tests that use fork(), that means emulated
fork() (with ithreads) under Windows, and they've been working fine
for years.
2) DBIS/dTHR issue was fixed at DBD::SQLite 1.22_04 (released in
April, 2009) by Tim's advice.
3) Docs on SQLite3's thread safety:
- http://www.sqlite.org/compile.html#threadsafe
- http://www.sqlite.org/c3ref/threadsafe.html
- http://www.sqlite.org/faq.html#q6
4) DBD::SQLite is compiled with the default THREADSAFE option if your
perl supports ithreads, or THREADSAFE=0 (single thread mode)
otherwise.
5) As a general rule, a) you shouldn't use DBI handles prepared
outside the threads, and b) you're advised to set
sqlite_use_immediate_transaction attribute to true (or issue BEGIN
IMMEDIATE) to avoid (dead)locks.
Regards,
Kenichi
(Sorry for dupe, Tim)
2012/3/19 Tim Bunce :
> On Mon, Mar 19, 2012 at 08:42:04AM +, Martin J. Evans wrote:
>> On 19/03/12 04:19, Adam Kennedy wrote:
>> >I've noticed there's a lot of movement at the moment on DBI, threading
>> >and performance.
>> >
>> >http://www.martin-evans.me.uk/node/131
>>
>> The issue here was that when using a threaded Perl the state structure
>> had to be protected and there was a slow and a faster way of accessing
>> it. Some DBDs were using the slow method.
>
> ("protection" isn't the issue, it's simply that in a threaded perl
> there's one 'DBI State' structure per thread and it's important that the
> one for the current thread is used.)
>
>> >In Padre we've always stuck to the use of DBI only in the parent
>> >thread, but the time is fast approaching where it would be very handy
>> >to run multiple database connections for things like background
>> >indexing of code and the like.
>>
>> When you build DBI it still says:
>>
>> *** You are using a perl configured with threading enabled.
>> *** You should be aware that using multiple threads is
>> *** not recommended for production environments.
>>
>> but I'm unsure to what degree this still applies.
>
> I think that could be removed, at least for recent perls.
> I wonder if there's some version where a good number of thread issues
> got fixed. Perhaps 5.10, 5.12? Would anyone using thread havily care to
> venture an opinion?
>
>> >I was wondering if anyone has any experiences with DBD::SQLite and
>> >threads, or can speak with some authority on where we are with regards
>> >to them (I do know that the CLONE method seems to blank out the driver
>> >structure forcing it to load again in the new thread).
>
> I think that's needed. Certainly handles created in one thread can't be
> used in another (the DBI dispatcher will detect that and throw an error).
>
>> >Regardless of whether they currently work or not, I'd like to be able
>> >to write a =head2 section in the POD documentation stating our current
>> >position on threads and recommendations before the next release.
>> >
>> >If they can be used in threads, it would be nice to be able to write a
>> >DBD::SQLite::Cookbook entry on using threads.
>> >Adam K
>>
>> I'm afraid I neither use a Perl with threads enabled or threads -
>> which is a shame as I didn't see any benefit from that huge change to
>> DBD::Oracle.
>
> Many people will benefit. Thanks again.
>
>> I note DBD::SQLite does not seem to be using the DBIS macro so there
>> is nothing to change.
>
> Agreed. It looks clean to me.
>
> Tim.
>
> ___
> DBD-SQLite mailing list
> DBD-SQLite@lists.scsys.co.uk
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] [rfc][patch] fix >32bit integer truncation on some machines [2/2]
Applied both to the trunk.
Thanks,
Kenichi
2012/3/24 Yuriy Kaminskiy :
> Yuriy Kaminskiy wrote:
>> This cannot be right:
>>
>>> sqlite_set_result(pTHX_ sqlite3_context *context, SV *result, int is_error)
>>> {
>>> ...
>>> else if ( SvIOK(result) ) {
>>> sqlite3_result_int( context, SvIV(result));
>>> ...
>
> ... and this may truncate integer to 32-bit on some machines. Untested [my
> perl
> build unaffected] patch (on top of alternative 2, otherwise apply by hand)
> attached.
>
> ___
> DBD-SQLite mailing list
> DBD-SQLite@lists.scsys.co.uk
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
>
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] [patch][2/2] Convert unsigned -> int64 when possible
Applied the last three (two in this thread and one in the previous thread) to the trunk, either. Thanks, Kenichi 2012/3/25 Yuriy Kaminskiy : > We can avoid converting UV to text when it fits in int64. > Test fails before patch only on machines with 32-bit IV/UV. > > ___ > DBD-SQLite mailing list > DBD-SQLite@lists.scsys.co.uk > http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite > ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] DBD:SQlite with microperl?
Hi.
I don't think you can install DBD::SQLite with microperl. If you need
to use microperl, and you have a sqlite3 binary at hand, you'll
probably want to use system() (or backticks `` to get values):
./microperl -e 'system(q{ /path/to/sqlite3 db.file "some sql statement" })'
Refer to sqlite3 manual for details.
Regards,
Kenichi
2012/7/25 Dimitar Penev :
> Hi Guys,
>
> I am new to Perl.
>
> I have ported both microperl and SQLite on an embedded board based on
> Blackfin/uClinux.
>
> Do you think I can install DBD:SQlite easily or does it require the full
> Perl?
>
> Thank you.
> Dimitar
>
> ___
> DBD-SQLite mailing list
> DBD-SQLite@lists.scsys.co.uk
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] might Alien::Base be useful to DBD::SQLite development?
2013/5/11 Darren Duncan : > I remember, a few years ago around when this mailing list started, there was > some discussion about that one way to help make DBD::SQLite development > easier was to make use of something called "alien", a project to help use C > libraries from Perl without having to explicitly code XS or wrappers to do > so, and maybe pull it from the internet too, or something like that. No, Alien::* is not about coding. It's about locating (and building if not available) external C libraries/header files and gathering information for configuration. You still need to write XS code. As for DBD::SQLite, the SQLite3 source is bundled in the distribution, and thus, Alien is not necessary (we don't need to locate the stuff), unless you'd like to use different versions of SQLite from the bundled one, and we have disabled that option intentionally in the Makefile.PL for several reasons. - K > I see now that this project may have been realized to the point that we can > try using it for DBD::SQLite at least experimentally: > > http://blogs.perl.org/users/joel_berger/2013/05/alienbase-final-report.html > > So if there are any interested parties, there's something for you to play > with. > > -- Darren Duncan > > ___ > DBD-SQLite mailing list > DBD-SQLite@lists.scsys.co.uk > http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
[DBD-SQLite] repository moved
Hi. 1) DBD::SQLite repository is moved to https://github.com/DBD-SQLite/DBD-SQLite (sorry for the late announcement). 2) As for the next stable version, I found a blocker; it looks like DBD::SQLite 1.38_02/03 doesn't compile well under the latest *BSD environment with clang support. I'm setting up FreeBSD 9.1 VM now to see what's actually happening, but any help would be appreciated. Kenichi Ishigaki aka charsbar ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] SQLite compiled without SQLITE_ENABLE_COLUMN_METADATA
Hi.
It looks like those two errors come from different reasons. The first
one (of t/51...) can be safely skipped if DBD::SQLite is not compiled
with ENABLE_COLUMN_METADATA (see t/rt_40594...). You probably need to
update your manifest or run util/constants.pl (in the repository)
before you make a distribution to address the second.
Regards,
Kenichi
2015-10-01 0:16 GMT+09:00 Vladimir Marek :
> Hi,
>
> I am working on integrating DBD::SQLite into Solaris. I need it to link
> to /usr/lib/libsqlite3.so . The problem is that the library is compiled
> without SQLITE_ENABLE_COLUMN_METADATA. Is there any experience with this
> combination?
>
> I can make DBD::SQLite compile by removing the define in question from
> Makefile.PL. Running test shows some problems afterward:
>
> t/51_table_column_metadata.t .. 1/33
> # Failed test 'id is auto incremental'
> # at t/51_table_column_metadata.t line 21.
>
> # Failed test 'data type is correct'
> # at t/51_table_column_metadata.t line 22.
> # got:''
> # expected: 'integer'
>
> # Failed test 'id is a primary key'
> # at t/51_table_column_metadata.t line 23.
>
> # Failed test 'data type is not defined'
> # at t/51_table_column_metadata.t line 31.
> # got: '�('
> # expected: undef
>
> # Failed test 'id is auto incremental'
> # at t/51_table_column_metadata.t line 21.
>
> # Failed test 'data type is correct'
> # at t/51_table_column_metadata.t line 22.
> # got:''
> # expected: 'integer'
>
> # Failed test 'id is a primary key'
> # at t/51_table_column_metadata.t line 23.
>
> # Failed test 'data type is not defined'
> # at t/51_table_column_metadata.t line 31.
> # got: '�('
> # expected: undef
> # Looks like you failed 8 tests of 33.
>
>
> and
>
>
> t/virtual_table/11_filecontent_fulltext.t . 2/15
> # Failed test 'search '"use strict" AND "use warnings"''
> # at t/virtual_table/11_filecontent_fulltext.t line 79.
> # Structures begin differing at:
> # $got->[1] = 'lib/DBD/SQLite/VirtualTable.pm'
> # $expected->[1] = 'lib/DBD/SQLite/Constants.pm'
>
> # Failed test 'search '"use strict" AND "use warnings"' -- no
> # lib/DBD/SQLite/VirtualTable.pm'
> # at t/virtual_table/11_filecontent_fulltext.t line 92.
> # Structures begin differing at:
> # $got->[1] = 'lib/DBD/SQLite/VirtualTable/FileContent.pm'
> # $expected->[1] = 'lib/DBD/SQLite/Constants.pm'
>
> # Failed test 'search '"use strict" AND "use warnings"' -- after
> # reconnect'
> # at t/virtual_table/11_filecontent_fulltext.t line 105.
> # Structures begin differing at:
> # $got->[1] = 'lib/DBD/SQLite/VirtualTable/FileContent.pm'
> # $expected->[1] = 'lib/DBD/SQLite/Constants.pm'
> # Looks like you failed 3 tests of 15.
>
>
> and
>
>
> t/virtual_table/11_filecontent_fulltext.t
>
>
> Thank you
> --
> Vlad
>
> ___
> DBD-SQLite mailing list
> DBD-SQLite@lists.scsys.co.uk
> http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
___
DBD-SQLite mailing list
DBD-SQLite@lists.scsys.co.uk
http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] SQLite compiled without SQLITE_ENABLE_COLUMN_METADATA
2015-10-06 7:15 GMT+09:00 Vladimir Marek :
> Thank you for your quick reply.
>
>> It looks like those two errors come from different reasons. The first
>> one (of t/51...) can be safely skipped if DBD::SQLite is not compiled
>> with ENABLE_COLUMN_METADATA (see t/rt_40594...).
>
> For the t/51_table_column_metadata.t I intend to exclude only the failed
> tests. But I am struggling with the second part of problems.
>
>> You probably need to update your manifest or run util/constants.pl (in
>> the repository) before you make a distribution to address the second.
>
> I'm a bit struggling with what is going on here. I have downloaded the
> constants.pl and SQLiteUtil.pm script, fiddled with it so that it
> consumes /usr/include/sqlite3.h, but the files produced by it
> (contstants.inc and Constants.pm) are the same as the ones in the
> DBD-SQLite-1.48.tar.gz .
>
> I then went into the test itself and it seems that it's generating full
> text index of all the files contained in the MANIFEST file. Then it's
> looking for given words and compares the output with predetermined
> 'correct' set. All tests work, but one. Looking for
> '"use strict" AND "use warnings"' does not find
> lib/DBD/SQLite/Constants.pm. I changed the test and I can confirm that
> looking for both '"use warnings"' and '"use strict"' finds that file.
> I also verified which file is opened by the test and I made sure that
> both strings are present. I have tried to dump the sqlite database, but
> it's mostly full of binary blobs.
>
> I know next to nothing about DBD::SQLite and/or SQLite itself, but the
> guts feeling is that the perl module is just a wrapper, so maybe SQLite
> itself could be blamed here? We have version 3.8.8.1. Looking here:
> https://sqlite.org/news.html does not reveal any relevant bug being
> fixed.
>
> I can compile my own sqlite and link the perl module to it, but at the
> moment that's all I can think of ...
Are you using a custom packager script, or making DBD::SQLite all by hand?
If the former is the case, check your packaging script and see if
there's a list of extracted file to be copied to somewhere (in a
temporary directory etc) and lib/DBD/SQLite/Constants.pm is correctly
contained in the list. The file is added fairly recently (April 2015),
so if you are using a script written some time earlier, the file may
not be copied correctly, which leads to your error.
According to the CPAN Testers, DBD::SQLite normally passes all its
tests under Solaris. So I don't think there's anything bad in the code
itself.
http://matrix.cpantesters.org/?dist=DBD-SQLite+1.48
Regards,
Kenichi
>
> Thank you!
> --
> Vlad
>
>
>
>>
>> Regards,
>> Kenichi
>>
>>
>> 2015-10-01 0:16 GMT+09:00 Vladimir Marek :
>> > Hi,
>> >
>> > I am working on integrating DBD::SQLite into Solaris. I need it to link
>> > to /usr/lib/libsqlite3.so . The problem is that the library is compiled
>> > without SQLITE_ENABLE_COLUMN_METADATA. Is there any experience with this
>> > combination?
>> >
>> > I can make DBD::SQLite compile by removing the define in question from
>> > Makefile.PL. Running test shows some problems afterward:
>> >
>> > t/51_table_column_metadata.t .. 1/33
>> > # Failed test 'id is auto incremental'
>> > # at t/51_table_column_metadata.t line 21.
>> >
>> > # Failed test 'data type is correct'
>> > # at t/51_table_column_metadata.t line 22.
>> > # got:''
>> > # expected: 'integer'
>> >
>> > # Failed test 'id is a primary key'
>> > # at t/51_table_column_metadata.t line 23.
>> >
>> > # Failed test 'data type is not defined'
>> > # at t/51_table_column_metadata.t line 31.
>> > # got: '�('
>> > # expected: undef
>> >
>> > # Failed test 'id is auto incremental'
>> > # at t/51_table_column_metadata.t line 21.
>> >
>> > # Failed test 'data type is correct'
>> > # at t/51_table_column_metadata.t line 22.
>> > # got:''
>> > # expected: 'integer'
>> >
>> > # Failed test 'id is a primary key'
>> > # at t/51_table_column_metadata.t line 23.
>> >
>> > # Failed test 'data type is not defined'
>> > # at t/51_table_column_metadata.t line 31.
>> > # got: '�('
>> > # expected: undef
>> > # Looks like you failed 8 tests of 33.
>> >
>> >
>> > and
>> >
>> >
>> > t/virtual_table/11_filecontent_fulltext.t . 2/15
>> > # Failed test 'search '"use strict" AND "use warnings"''
>> > # at t/virtual_table/11_filecontent_fulltext.t line 79.
>> > # Structures begin differing at:
>> > # $got->[1] = 'lib/DBD/SQLite/VirtualTable.pm'
>> > # $expected->[1] = 'lib/DBD/SQLite/Constants.pm'
>> >
>> > # Failed test 'search '"use strict" AND "use warnings"' -- no
>> > # lib/DBD/SQLite/VirtualTable.pm'
>> > # at t/virtual_table/11_filecontent_fulltext.t line 92.
>> > # Structures begin differing at:
>> > # $got->[1] = 'lib/DBD/SQLite/VirtualTable/FileContent.pm'
>> > # $expected->[1] = 'lib/DBD/SQLite/Constants.p
Re: [DBD-SQLite] policy for FTS5 integration ?
Hi, thanks for starting the discussion :) As of 3.9.0, SQLite compiles flawlessly with both fts3/4 and fts5 enabled. So it looks like we don't need to choose one. If some of you want to try the experimental fts5 right now, I'll add SQLITE_ENABLE_FTS5 to Makefile.PL. Otherwise, I'll keep it disabled until it gets stable. Cheers, Kenichi Ishigaki 2015-10-26 16:30 GMT+09:00 Darren Duncan : > I believe the best option is d) to support both FTS4 and FTS5 at the same > time, as if they were different data types. Failing that, I like b) the > best. I also think a) is the worst option; this is not something worth > forking over. -- Darren Duncan > > On 2015-10-25 11:46 PM, Dami Laurent (PJ) wrote: >> >> Hi there, >> >> I noticed that the 3.9.0 amalgamation now contains FTS5. So it’s time to >> think >> about the evolution of FTS support in DBD::Sqlite. >> >> At the moment, FTS5 is still experimental, but at some point it will >> stabilize >> and become mainstream. >> >> FTS5 contains incompatible changes from FTS3/4 (see >> http://www.sqlite.org/fts5.html#appendix_a). >> >> Some of those incompatibilities can be handled in the DBD::Sqlite driver >> and >> therefore will be invisible to users ; but some other points change the >> SQL API >> to FTS and therefore **will** be visible. >> >> So what should be the future evolution of DBD::Sqlite with respect to FTS >> ? Here >> are some options : >> >> a)Stick with FTS4 forever to preserve backwards compatibility, and fork a >> new >> distrib DBD::SQLite::FTS5 (but then how to keep both distribs in sync ?) >> >> b)Target for a backwards-incompatible move to FTS5 at some point in the >> future, >> and advertise it long in advance >> >> c)Support 2 variants, with an option in Makefile.PL to choose the FTS >> version at >> compilation time (but this could be a nightmare for automatic toolchains). >> >> d)Try to simultaneously activate FTS3 and FTS5 in the same compilation (I >> don’t >> know if this is possible), and let users choose through runtime options. >> >> e)Any other idea ? >> >> DBD::Sqlite never encountered this issue before, because the first FTS >> support >> was for FTS3, and then when sqlite.org introduced FTS4 it was fully >> compatible >> with FTS3. The only somewhat similar situation was in2010 when the FTS >> syntax >> changed from « Standard Query Syntax » to « Extended Query Syntax » ; at >> that >> time I published the DBD::Sqlite::FTS3Transitional module to help with the >> migration, but this time it will not be that simple. >> >> So what do you think ? My personal preference would be for option b). It’s >> not >> very nice to break backwards compatibility, but sometimes it must happen. >> Besides, I never saw any public discussions, comments or bug reports about >> FTS >> support in DBD ::Sqlite, so the user community is probably quite small, >> and >> therefore the impact of such a change would be limited. Of course this >> should >> only be activated when FTS5 has becomes stable, which should still take a >> couple >> of months. >> >> Once the policy issue is settled, I’m interested to volunteer for working >> on the >> technical aspects – but I have very little free time to do so, so I will >> not >> progress very fast. If anybody else has more time, they are welcome to >> take the job. >> >> Cheers, Laurent Dami > > > > > ___ > DBD-SQLite mailing list > DBD-SQLite@lists.scsys.co.uk > http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
Re: [DBD-SQLite] adding support ot busy handler support in DBD::SQLite
Hi 2017-07-12 5:51 GMT+09:00 George Shimanovich : > Hello, > > > Current implementation of DBD::SQLite supports busy timeout > (sqlite_busy_timeout), but not busy handler. > > > sqlite3's busy handler (sqlite3_busy_handler) offers alternative way to deal > with busy timeout (sqlite3_busy_timeout). > > For example, it allows one to implement infinite busy timeout with a > callback used to identify (e.g. once a minute) process holder of write lock > to the database. With busy timeout the caller needs to implement retry logic > for every query, which is cumbersome. > > > Are there plans to add support to busy handler? > > > I am thinking of implementing driver private method sqlite_busy_handler and > will welcome any insights. Perhaps I am missing something. > Pull requests are always welcome :) Cheers, Kenichi > > I considered using sqlite_progress_handler but can't implement the above > busy handler functionality using it, as there is no way to determine that > database is busy. Plus sleeping in progress handler defeats its purpose and > causes delay. > > > Thank you, > > George > > > ___ > DBD-SQLite mailing list > DBD-SQLite@lists.scsys.co.uk > http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite > ___ DBD-SQLite mailing list DBD-SQLite@lists.scsys.co.uk http://lists.scsys.co.uk/cgi-bin/mailman/listinfo/dbd-sqlite
