Re: [sqlite] [sqlite-dev] Poor SQLite indexing performance

2010-03-16 Thread Tim Romano
I have not done this, but if you have enough RAM available, you might try putting your primary keys in a table in an in-memory database, and test for existence there. That would allow you to enforce uniqueness while postponing creation of the PK index on the disk table until after the initial

[sqlite] Unicode command line bug in Windows version of sqlite3 with patch

2010-03-16 Thread Kunc Filip
Hello SQLite Team, We currently use sqlite 3.6.23. We have a big problem with characters with accents or other special characters in path to database file, for example in Czech Windows XP the "Application Data" folder is translated to "Data aplikací" so if the accented 'í' is in path the

[sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread HLS
Folks, I read the documentation suggesting to use another SQL engine for a multi-threaded client/server environment and I reached a point where I see why.   But I was thinking it could be used because currently, we have a ISAM/BTREE database under our full reader/writer and exclusive locking

Re: [sqlite] Unicode command line bug in Windows version of sqlite3 with patch

2010-03-16 Thread Pavel Ivanov
sqlite3_open[_v2] accepts all filenames in UTF-8 (although it doesn't check for valid UTF-8 string). So CP_UTF8 cannot be changed anywhere. OTOH maybe command line utility should have some logic of re-encoding of command line parameter from terminal encoding to UTF-8. But I'm not sure about that.

Re: [sqlite] Porting Sqlite to MQX OS: Question 2

2010-03-16 Thread GeoffW
Hello Dan >Do the tasks share a heap? And do you have multiple connections >to the database (multiple calls to sqlite3_open() or sqlite3_open_v2(). I would like to keep the implementation as general purpose as possible so we can use sqlite in the future in differient Applications. Therefore I

Re: [sqlite] Unicode command line bug in Windows version of sqlite3 with patch

2010-03-16 Thread Jean-Christophe Deschamps
>We currently use sqlite 3.6.23. We have a big problem with characters with >accents or other special characters in path to database file, for >example in >Czech Windows XP the "Application Data" folder is translated to "Data >aplikací" so if the accented 'í' is in path the sqlite3.exe writes

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread Pavel Ivanov
I didn't understand how you make everything to work, but can answer at the following question: > So how do I make it so there is no locking? I guess you already know that there's concept of VFS in SQLite (http://www.sqlite.org/c3ref/vfs.html). In VFS there's structure sqlite3_io_methods

Re: [sqlite] VACUUM & journal size

2010-03-16 Thread Max Vlasov
> This means that to VACUUM a SQLite database of size X, you need at > least 2X of _additional_ free disk space available. That seems rather > wasteful, just looking at it as a SQLite user. Although > programmatically there may be reasons for it that I'm not aware of. > > Hmm, did some

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread HLS
Pavel, thanks for your response. I will read up again and pay more attention to the technical VFS details (try it out).To answer your question, maybe I am beating a dead horse. :) This is an exploratory project. Right now, the server ISAM/BTREE is open in exclusive mode (no process

Re: [sqlite] VACUUM & journal size

2010-03-16 Thread Pavel Ivanov
> So the question is what is so special about sqlite3RunVacuum that it needs > more space than a simple emulation of its actions? I believe renaming of the file cannot be atomic. So in case of OS crash you can be in situation without database at all - no old and no new. Also deleting of old file

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread Olaf Schmidt
"HLS" schrieb im Newsbeitrag news:9cf5931e1003160705v38335b5g47a3d91193c28...@mail.gmail.com... > ...we have a ISAM/BTREE database under our full > reader/writer and exclusive locking controls... >From your posted code-snippet I don't see any "heavy SQL-Query-usage" - so,

Re: [sqlite] VACUUM & journal size

2010-03-16 Thread Jay A. Kreibich
On Tue, Mar 16, 2010 at 06:18:13PM +0300, Max Vlasov scratched on the wall: > When I read the comments it was obvious that the algorithm uses very simple > approach: > Attach blank database, copy all data, detach, rename. Sure I might be > wrong in details, but generally it looks like this.

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread Pavel Ivanov
> But > SQLITE3 locks the whole request.  It would be seem to me that this > could be an improvement to release locks at the record/row level (once > the fetch was done).  But then again, that is probably what VFS is > there for. SQLite locks the whole database when reading cursor is open because

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread HLS
On Tue, Mar 16, 2010 at 11:54 AM, Olaf Schmidt wrote: > > "HLS" schrieb im > Newsbeitrag > news:9cf5931e1003160705v38335b5g47a3d91193c28...@mail.gmail.com... > >> ...we have a ISAM/BTREE database under our full >> reader/writer and exclusive locking

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread HLS
Thanks Pavel, it adds a lot of weight to the reality of where SQLITE3 fits. On Tue, Mar 16, 2010 at 12:04 PM, Pavel Ivanov wrote: >> But >> SQLITE3 locks the whole request.  It would be seem to me that this >> could be an improvement to release locks at the record/row level

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread HLS
On Tue, Mar 16, 2010 at 1:01 PM, HLS wrote: > >   dim f as TFileRecord >   dim tid as integer >   dim n as integer = 0 >   LoginSystem() >   if GetFirstFileRec(FileNameAreaKey, f, tid) then >      do >       inc(n) >      ... do something ... >      loop while

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread Simon Slavin
On 16 Mar 2010, at 5:17pm, HLS wrote: > Once approach is to queue any updates/deletes when the database is > locked with a select request. So when a fetch ends (like in the > GetNext function), it will check to see for any pending updates > and process them. > > Does that sound like a

[sqlite] question regarding performance tuning of HTML5 Db (Sqlite)

2010-03-16 Thread p r
Folks, Couple of questions we were wrestling with: 1. How to tune performance of the application that uses HTML5 DB (Sqlite under the covers). Any general tips or suggestions e.g. is it possible to use in memory sqlite db or attach multiple db's (some in memory dbs some regular on disk dbs?

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread HLS
Thanks Simon.  It just seem so simplistic that SQLITE3 does not allow for Open Cursor for each fetch     Issue Update based on ROWID endfor Close Cursor The row fetched is already complete, or the rowid in the table is no longer "sensitive" to anything but a update whether it was opened or not.

Re: [sqlite] SQLite file format

2010-03-16 Thread Magnus Torfason
On 3/11/2010 10:46 AM, Diana Chan wrote: > I have some questions about SQLite. I would like to know if it's possible > to use SQLite as storage for huge genomic datasets. These datasets are > currently in netcdf format. I'm wondering if it's possible to convert them > to the SQLite file saved

[sqlite] Backuping SQLite database in VC system

2010-03-16 Thread Alberto Simões
Hello I was pondering to backup an SQLite database using a version control system (probably SVN). Of course that if I add the binary file SVN will diff the entire file. That is not a good option. I though on dumping the full database to SQL and put that file into svn. I would like to ask: -

Re: [sqlite] Backuping SQLite database in VC system

2010-03-16 Thread Alberto Simões
Hello 2010/3/16 Alberto Simões : > Hello > > I was pondering to backup an SQLite database using a version control > system (probably SVN). Of course that if I add the binary file SVN > will diff the entire file. That is not a good option. > > I though on dumping the full

Re: [sqlite] Backuping SQLite database in VC system

2010-03-16 Thread Darren Duncan
Alberto Simões wrote: > I was pondering to backup an SQLite database using a version control > system (probably SVN). Of course that if I add the binary file SVN > will diff the entire file. That is not a good option. > > I though on dumping the full database to SQL and put that file into svn. >

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread Pavel Ivanov
Please be advised that updating/inserting/deleting from the table while you're executing select on it has undefined behavior in SQLite (if you do that on the same connection) and is considered dangerous. Doing that from different connections is possible only if have shared cache turned on and

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread Simon Slavin
On 16 Mar 2010, at 6:12pm, HLS wrote: > It just seem so simplistic that SQLITE3 does not allow for > > Open Cursor > for each fetch > Issue Update based on ROWID > endfor > Close Cursor One reason you cannot do this is that changing the value of a field may change how you step from row to

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread HLS
Do you know if TLS is used? How does it workout splite3_open_v2() per thread? Can the handle be used globally? In other words, is open per thread considered a "different connection?" On Tue, Mar 16, 2010 at 3:57 PM, Pavel Ivanov wrote: > Please be advised that

Re: [sqlite] Unicode command line bug in Windows version of sqlite3 with patch

2010-03-16 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Jean-Christophe Deschamps wrote: > A much better solution is to use a MSYS terminal (installed by MinGW), > so you have UTF-8 command-line and data entry/display without > conversion. No need to "patch" anything. No need for msys. You can make a

Re: [sqlite] SQLITE3 in multi-thread server

2010-03-16 Thread HLS
On Tue, Mar 16, 2010 at 4:44 PM, Simon Slavin wrote: > One of the things that makes SQLite very simple is that it doesn't try to do > this, it has either everything or nothing locked. Yes, that is coming to realization now. -- hls

Re: [sqlite] Attach a database dynamically if not already available within a trigger

2010-03-16 Thread Kees Nuyt
On Mon, 15 Mar 2010 18:52:26 -0400, andy wrote: >Hello >I am hoping someone can help with the following problem. > >- I want to create a log of all changes to tables in a database. >- I would like to keep the log table in a separate database to the >main data files >-

[sqlite] Datatypes wiki note

2010-03-16 Thread Dennis Volodomanov
Hello all, A small note - on this page: http://sqlite.org/datatype3.html it says CAST(expr TO type), but it should be CAST(expr AS type), right? Best regards, Dennis ___ sqlite-users mailing list sqlite-users@sqlite.org

[sqlite] SQLite on 64-bit Windows O/S

2010-03-16 Thread Rashed Iqbal
Has anyone compiled and used SQLite on 64-bit Windows? Are there any code changes that would be needed before compiling? Would the same DB file work on 32-bit and 64-bit Windows platforms? Thank you for your help. Rashed ___ sqlite-users mailing list

[sqlite] state of wrappers

2010-03-16 Thread Jay A. Kreibich
I'm trying to put together some documentation and need some help from SQLite users utilizing non-C languages. The wiki page on SQLite wrappers is a bit of a mess, so I'm hoping some people that actually use these languages can offer some opinions. In specific, I'm trying to understand

Re: [sqlite] SQLite on 64-bit Windows O/S

2010-03-16 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Rashed Iqbal wrote: > Has anyone compiled and used SQLite on 64-bit Windows? Are there any > code changes that would be needed before compiling? Would the same DB > file work on 32-bit and 64-bit Windows platforms? The file format is fixed and is

Re: [sqlite] SQLite on 64-bit Windows O/S

2010-03-16 Thread Random Coder
On Tue, Mar 16, 2010 at 6:21 PM, Rashed Iqbal wrote: > Has anyone compiled and used SQLite on 64-bit Windows? Are there any > code changes that would be needed before compiling? Would the same DB > file work on 32-bit and 64-bit Windows platforms? I routinely run SQLite on