Re: [sqlite] .import error: cannot open large file
Hello, On Fri, Jun 3, 2011 at 00:08, Rense Corten <rcor...@gmail.com> wrote: [...] > So I searched the archives of this list and found two threads on this: > > http://www.mail-archive.com/sqlite-users@sqlite.org/msg51574.html > http://www.mail-archive.com/sqlite-users@sqlite.org/msg48649.html > > The first thread got no answers, but the second suggest to either > split the file or recompile sqlite3 with the option for large file > support . Now I had understood that since version 3.5.9 large file > support is switched on by default so that should not be the problem > (http://www.sqlite.org/changes.html). Splitting the file, however, > seems to solve the problem. I would prefer not to have to split the > file first. > > Any ideas on what causes this problem? It's what the thread says. The SQLite shell on Ubuntu (on 11.04) isn't compiled with large file support on 32-bit systems, so while the SQLite library does work with 64-bit database, the shell doesn't. The easy solution is to either use a 64-bit Ubuntu system or compile the shell yourself with large file support. To compile it, download the sqlite amalgamation files and run: gcc -o sqli -O3 -DNDEBUG=1 -D_FILE_OFFSET_BITS=64 sqlite3.c shell.c -ldl -pthread The resulting binary (sqli) will be compiled with large file support (I verified it was using strace). Regards, ~Nuno Lucas P.S.- While this could be considered an Ubuntu bug, the truth is that the linux shell binary on the sqlite site also isn't compiled with large file support, so I would consider this an SQLite bug. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] .import error: cannot open large file
On Fri, Jun 3, 2011 at 18:28, Rense Corten <rcor...@gmail.com> wrote:
> Thanks for your answer, Nuno.
> However, the system I am using is already 64-bit (I should have
> mentioned that) , and the same binary can do the job on another Ubuntu
> 64-bit system. I'll try your suggestion nevertheless, but can there
> be other causes?
> Rense
There can always be other causes, like permission problems. But from
what you say I'm tended to believe your problem is the one I
mentioned.
Note that although your system is 64-bits, the pre-compiled sqlite
binary is 32-bits.
But you can check yourself using "strace".
Run the shell as:
$ strace -e open sqlite ...args...
You will see all open system calls, and for you to be able to open
large files you must see something like:
open("file.csv", O_RDONLY|O_LARGEFILE) = 4
Notice the O_LARGEFILE flag. If the shell is not compiled for large
file access that flag will not be present.
The latter is what happens using the pre-compiled sqlite binary.
Regards,
~Nuno Lucas
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Table info for a particular query
> Is there any way to do something along the lines of PRAGMA > some_command(SELECT * FROM table1 JOIN table2) and have it return A, B, C, > a, b, c as the headers? If you really want to do it with "table_info", you could create a temporary view with the "select": SQLite version 3.6.17 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> create table t1 ( A, B, C ); sqlite> create table t2 ( a, b, c ); sqlite> create temp view v1 as select * from t1 join t2; sqlite> .mode col sqlite> .h 1 sqlite> insert into t1 values ( 1, 2, 3 ); sqlite> insert into t1 values ( 2, 3, 4 ); sqlite> insert into t2 values ( 3, 4, 5 ); sqlite> insert into t2 values ( 4, 5, 6 ); sqlite> select * from v1; A B C a:1 b:1 c:1 -- -- -- -- -- -- 1 2 3 3 4 5 1 2 3 4 5 6 2 3 4 3 4 5 2 3 4 4 5 6 sqlite> pragma table_info(v1); cid nametypenotnull dflt_value pk -- -- -- -- -- -- 0 A 0 0 1 B 0 0 2 C 0 0 3 a:1 0 0 4 b:1 0 0 5 c:1 0 0 ---- But I'm not sure if it's exactly the same as if from a select. Regards, ~Nuno Lucas ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] c89 (long long) compatibility warning in amalgamation build
Hello, On Thu, Sep 8, 2011 at 11:32, Stephan Beal <sgb...@googlemail.com> wrote: [...] > The compile warning says it all: > > [stephan@cheyenne:~/cvs/fossil/whiki]$ make > + cc -pedantic -Wall -Werror -fPIC -std=c89 -g -I. -I. -I./include > -DSQLITE_THREADSAFE=0 -DSQLITE_TEMP_STORE=3 -DSQLITE_OMIT_LOAD_EXTENSION=1 > -c -o sqlite3.o sqlite3.c > cc1: warnings being treated as errors > sqlite3.c:801: error: ISO C90 does not support 'long long' > sqlite3.c:802: error: ISO C90 does not support 'long long' > make: *** [sqlite3.o] Error 1 > > [stephan@cheyenne:~/cvs/fossil/whiki]$ sed -n 800,803p sqlite3.c > #else > typedef long long int sqlite_int64; > typedef unsigned long long int sqlite_uint64; > #endif > > i unfortunately cannot propose a portable alternative. > > This probably isn't going to bother most people, but in a recent discussion > it was mentioned that c89 mode should work for fossil (and, by > extension/implication, sqlite), if only to maximize portability. As far as I know, there is no portable way of defining 64-bit integers on ISO C90, so, by definition, it's a mission impossible. Only C99 come with the «stdint.h» header which would define the "int64_t" and "uint64_t" types, if supported by the platform (one way to check is to verify if the INT64_MAX macro is defined). Off course you can always use "#ifdef"s to workaround this limitation on C90, but no "true answer" (TM) exists. If I remember correctly, you can use the header in C89 mode with the Linux libc, so you could workaround with that in mind. Regards, ~Nuno Lucas ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Encoding and Collation
On Sun, Sep 11, 2011 at 15:55, Antonio Maniero <mani...@klip.net> wrote: >> >> I see. Well, SQLite2 is ancient: that ship has sailed and it's not coming >> back. >> >> Did SQLite2 actually implement case-insensitive comparison on accented >> Latin characters? I honestly don't know - by the time I got involved with >> SQLite (in late 2005), SQLite2 was already history, and its original >> documentation doesn't seem to exist anymore. SQLite2 didn't support ISO-8859-1 (or 15 or whatever). It just ignored 8-bit characters so you could put whatever you wanted and get whatever you put there back. That means you could also put there UTF-8 text, and the "-16" API functions allowed to directly store/retrieve windows UCS2 Unicode strings (UTF-16 was only after XP, iirc), which SQLite automatically encoded/decoded to UTF-8 text. The problem was that there was no way for 3rd party applications to know what was the encoding being used. If I remember correctly, UTF-8 was still prefered, and some SQL functions acting on text only worked well if it was UTF-8 (like LENGTH). > Maybe someone else could say about the reason that SQLite dropped 8859 > encoding. > > Probably SQLite 2 had not case insensitive comparison on 8859 because it has > many encodings and locales, but implement it would be ease and simple. It didn't drop support. It just never had it anyway, although never caring about it. Nothing changed, except now the SQLite 3 API *EXPLICITLY* says the text functions require either UTF-8 or UTF-16, but nothing stops someone doing the same as with SQLite2 and store it's text as BLOBs. >> Version 3 keeps support for 8859? >> >> No, not really. But, again, it won't prevent you from storing 8859-encoded >> strings in the database, and installing a custom collation that understands >> them, if you are so inclined. Personally, I'd seriously consider switching >> to UTF-8. >> >> > I doubt it that it would be ease to storing 8859 without string functions > problems. The proper collation would be simple, of course, but probably I > would need to re-implement all string functions too. Am I wrong? Do you relly know that there is no 8859 encoding? The standards go from ISO-8859-1 to ISO-8859-16 and you would need to have collations for all of them and a way to let the user choose which one is the right one for them (including regional variations). There are so many things wrong with this kind of reasoning I prefer to not say more. > I can use utf8 but for me SQLite won't be lite anymore without a simple > utf8 implementation. Hopefully someone else could have a ready solution for > collation, otherwise I will do my own implementation. It will never be > correct, but it will be enough. My guess is that you are assuming your text is ISO-8859-1 (commonly called Latin-1). There are many problems with this, like for example the fact ISO-8859-15 -- Latin-9 -- being the replacement, with the € (euro) sign added, but Windows decided to invent it's own encoding and it's "Latin" it's not exactly the same (the Euro sign is the symbol which usually get's corrupted). > Am I the only user that need a lite implementation of SQLite with case > insensitive? Don't assume a case insensitive match it's easy if done right. Read about the Unicode collations and you will understand why -- there doesn't exist a single generic upper/lower case function that works for all. SQLite could have an erroneous and basic case insensitive match (already has a basic one, for 7-bit ASCII), but that would not solve nothing that isn't already solved by the correct solution, which is the use of the ICU extension. It's very easy to replace the SQLite functions with user-defined ones, so if someone wants to go the easy way (partial support for just the common western scripts) it's easy. And already done by many, if you search the mailing list. As a final note, SQLite 2 never had any support for ISO-8859-X collations, so you have no reason to believe SQLite 3 would have it. Regards, ~Nuno Lucas > > Thanks. ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Encoding and Collation
On 09/11/2011 08:27 PM, Antonio Maniero wrote: It's very easy to replace the SQLite functions with user-defined ones, so if someone wants to go the easy way (partial support for just the common western scripts) it's easy. And already done by many, if you search the mailing list. It's exactly what I'm looking for. It could be my mistake but I searched the list and I couldn't find it. If not asking too much, can you suggest better terms to use in my search? I haven't replied earlier because I also didn't found a search term that worked well (although I'm sure I read those mails). So I wanted to find the code I myself did for this (less than 100 lines of commented code), but couldn't find it (at least not in the time a new implementation from scratch would take). Anyway, I see now that you already found what you needed, and know how easy it is to create custom collations. Just wanted to add a small warning: the problem with using custom collations is that if you use them in indexes, then all applications that use the same database must also have the same collation for everything to work OK. Most of the time you don't have to worry about it, but do take that in mind. Regards, ~Nuno Lucas ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] Performance analysis of SQLite statements
On 4/12/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: You used to be able to compile with -DVDBE_PROFILE=1 to enable some special assembly-language instructions that would use hi-res timers on ix586 chips to provide the cycle counts needed to execute each instruction in a VDBE program. But I haven't used that feature in years so I don't know if it still works or not. If you are talking of the "rdtsc" instruction, then to work with current dual core CPUs the test must be "pinned" to a single CPU, or you could be reading TSC values from different CPU's. Other than that, as long as the code works on a "single-core" CPU, it should work on newer ones. Off course, there is that word: "should" ;-) Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Performance analysis of SQLite statements
On 4/12/07, Brandon, Nicholas (UK) <[EMAIL PROTECTED]> wrote: > > You used to be able to compile with -DVDBE_PROFILE=1 to enable some > > special assembly-language instructions that would use > hi-res timers on > > ix586 chips to provide the cycle counts needed to execute each > > instruction in a VDBE program. But I haven't used that feature in > > years so I don't know if it still works or not. > > If you are talking of the "rdtsc" instruction, then to work > with current dual core CPUs the test must be "pinned" to a > single CPU, or you could be reading TSC values from different CPU's. > > Other than that, as long as the code works on a "single-core" > CPU, it should work on newer ones. > > Off course, there is that word: "should" ;-) I would add that if using windows I recommend using QueryPerformanceCounter() instead of "rdtsc" as this function/call already handles the above mentioned issue which affects multiprocessor/multicore systems. In a previous job I did some performance metrics since I presumed it would be slow but remember being pleasantly surprised that it was as fast (circa 4 clock cycles to retrieve and store in integer) as using assembler code. This was using VS C++ 2003 compiler. I'm not aware of a similar call in Unix The "standard" similar calls on Linux are gettimeofday() and clock_gettime(). But they require (at least last time I checked) a system call, so maybe not as light as the windows call (I remember some talk for the new linux kernels to have a faster way of fetching the "timeofday" value, but don't know how that ended). On the other hand, they are usually enough, because when measuring times from userspace you already have some degree of uncertainty, like having a higher priority process slowing down the app you are running. I seem to remember something about QueryPerformanceCounter() not taking into account things like the new thermal management features of modern CPUs (like dynamic frequency scaling) and could give different results on different CPUs (the motherboard can include a high-res time source, but many don't, and some are just too slow to fetch a value). Regards, ~Nuno Lucas Regards Nick - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLite and memory usage
On 4/18/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I performed a simple experiment where i placed printf statements in the routines sqlite3FreeX and sqlite3MallocRaw. They seem to be the two lowest level routines in SQLite that allocate and deallocate memory. I redirected the output to a text file and imported it into Excel (this may have been the hard way). I then summed all of the numbers expecting the result to be zero. What I found was that at the end there was over a MB of memory still allocated. I can't believe this answer so are there other routines called that allocate and deallocate memory? SQLite is extensively tested for memory leaks, so I would doubt much. I don't see why you think why those functions you mentioned are the lower level functions. You are forgetting the sqlite3Realloc one and by looking at util.c: [...] ** The above APIs are implemented in terms of the functions provided in the ** operating-system interface. The OS interface is never accessed directly ** by code outside of this file. ** ** sqlite3OsMalloc() ** sqlite3OsRealloc() ** sqlite3OsFree() ** sqlite3OsAllocationSize() ** [...] So, it seems you are not using the lowest levels routines, nor counting the right high level ones. Also, SQLite has it's own memory leak detector, which you can enable to make sure there are no memory leaks (but off course slowing down your program a lot) Look at the file util.c. It should give you an idea. Regards, ~Nuno Lucas Should I have done this another way? Ray - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Reducing memory usage when reading a blob
On 4/19/07, Stan Bielski <[EMAIL PROTECTED]> wrote: On a similar note, are there any hacks available to write a blob to a file using a limited memory buffer? As far as I can tell, the sqlite API requires that a blob be read in its entirety, which is rather unfortunate when the blob is bigger than memory... The usual way is to store your BLOB in chunks yourself, so you can obtain parts of it in random order (and not exausting your memory). For example: CREATE TABLE blobs ( blob_id, blob_seq, blob_data ); Now you can retrieve your blob data by: SELECT blob_data FROM blobs WHERE blob_id=:ID: ORDER BY blob_seq; Disclaimer: I haven't tested the SQL, but you should get the idea. Regards, ~Nuno Lucas Thanks, -Stan - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Odd performance issue under Windows
On 4/26/07, John Elrick <[EMAIL PROTECTED]> wrote: Using two machines as an example, XP Home and XP Media Center. XP Home and Media Center with the pragma synchronous=OFF executes the test operation in under 2.0 seconds. Set to FULL the times are, respectively 118 seconds and 8 seconds. The test involves writing 1,000 rows to a table with no start/end transaction. The resulting file is 59kb in size. As each row has a "hard flush", we can see that each one takes about 118 mili-secs (118/1000s). That is a value well within the range of current normal hard disks (it depends alot on the disk rotation speed). 8 mili-seconds is a too low value, It is a value within the average read access time of current hard-disks (well, maybe not so current, but in the same order of magnitude). Too fast for a single write, so it's impossible that the flush was done. Write caching is enabled on both machines, therefore it is likely that we can eliminate Windows caching as a factor. At this point we have several distinct possibilities: 1. Hard drive caching. Both machines are relatively new, the Home machine is a two year old HP Pavilion desktop replacement. The Media Center machine is a year old HP Pavilion desktop replacement. I would tend to think that the caching capabilities of the two machines are close enough that 59kb of data shouldn't cause an order of magnitude difference. I could be wrong. I believe you are right. 2. Settings which cause Media Center to return control more optimistically than Pro or Home. In this case, there would be a hypothetically higher risk of data loss on the Media Center machine. However, the point of a synchronous=FULL, as I understand it, it to thoroughly verify that the data is "safe". It would be good to know if there is a way to defeat this safeguard and how to determine if it has been defeated. The thing is cheap IDE disks already lie to the OS saying they had flushed the buffers when they don't. Maybe Windows knows it can't depend on the drive and falls back to the lazy write scheme, (the default on Windows 2000). Maybe Windows XP is more conscious about this and tries to make sure it does it's best to flush the buffers (or maybe it's disk drivers are better). The only way for you to make sure is to install 2000/Media Center and XP on the same machine and test. SQLite always does the right thing. If the OS lies and says it completed the operation (probably because the hardware also lied to it), there's nothing SQLite can do. There is an interesting parallel around Win95 and Win2000 FLUSH performance: http://support.microsoft.com/kb/281281 And you seem to not have noticed this link: http://www.microsoft.com/technet/prodtechnol/windows2000serv/maintain/optimize/wperfch7.mspx Regards, ~Nuno Lucas 3. Settings which cause Media Center to be more aggressive about flushing its cache than Pro or Home. If this hypothesis is correct, Pro or Home would be putting a higher priority on services other than disk I/O. One final factor in my weighing is that performance reports I have read on the Wiki and other postings indicate that inserts outside of a transaction should still be closer to the single digit mark than triple digits. All of this having been said, all I am certain about at this time is that XP/Vista/Pro/Home appear to be an order of magnitude slower in returning control than Media Center and W2K. I don't know why. And that bothers me. John - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Sqlite 14 (cant open database)
On 5/3/07, Arjen Markus <[EMAIL PROTECTED]> wrote: Lloyd wrote: That is a very large number of files! I know FAT32 can not handle files larger than 2 GB, and I imagine there is limit on the number of files as well. Try creating some subdirectories and moving the files there. The problem is most likely in the filesystem. Just a note. FAT32 can handle 4GB files, but the original vfat linux driver (because of a design error) could only handle 2GB. Also many linux and windows utilities were, at first, not designed to handle 4GB files, so it's usually not safe to have more than 2GB files on FAT32 (some disk utilities could even trash your drive). I don't know if the recent vfat drivers were fixed to handle more than 2 GB files, though. Maybe no one considered it important until recently (only now you have FAT32 USB disks with more than 4GB space free). Best regards, ~Nuno Lucas Regards, Arjen - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] May I ask why the source distribution mechanism was changed starting with 3.3.14?
On 5/4/07, Ken <[EMAIL PROTECTED]> wrote: 100% concur with Dennis. Thanks again for a great product! +1 I couldn't said it better, maybe even in my native language ;-) Best regards, ~Nuno Lucas Dennis Cote <[EMAIL PROTECTED]> wrote: [EMAIL PROTECTED] wrote: > > Can somebody please explain to my how 2 files is less manageable > than 60? > > > Richard, I think part of the problem is simple inertia. Some people have developed a methodology for using the sqlite source files based on the previous arrangement. They may have patches they apply to the files, existing makefiles, etc. The new amalgamation file breaks those methods. It is not backwards compatible. They had a working method and your change has broken that method. It's not that they can't make it work with the amalgamation, but rather that they don't think they should have to change their processes unnecessarily. It is also often easier to work with the individual source files rather than the amalgamation. Loading, searching and scrolling in a editor are often easier with smaller files, and it's often handy to have several files open to different locations when studying the source, which can be difficult with a single file in many editors. I really think most people who are asking for the individual files would be better served using CVS to get a particular version (if they don't want to track the latest developments in the head versions). Unfortunately, you make this a little harder than it needs to be by not tagging your releases in CVS. The lack of tags requires users to find the release date and time from the website's timeline page before retrieving the files based on that date and time. It would be easier to just select a particular CVS release tag. Using CVS has one drawback for many Windows users though, it doesn't provide the preprocessed source files. They still need to install a unix like toolset such as MinGW/MSYS or Cygwin to use the makefiles which generate those files. This adds additional complexity. I have used all the available methods at various times for various reasons. Now, I most often use the files you have prepared and distribute through your website. It is simply easier for me to let you do the work. I think many others would like you to simply add the amalgamation to the set of files you distribute, rather than replacing the preprocessed source. It will be easier for them if you continue to do the work that you used to do. I would like to take this opportunity to thank you again for all the work you have put into sqlite. You have created a tremendous resource which makes many peoples lives at least a little easier and hence better. I hope the rewards have been worth the effort. Dennis Cote - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] porting sqlite3 to embeded os-----lock question
On 5/9/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I am porting sqlite3 to the embeded os,such as threadx,nucleus,ect.. I am writing the file such as os_threadx.c,os_nucleus.c according to the os_win.c,os_unix.c. I have read the os_win.c and find that there is a switcher OS_WINCE in the struct winFile. Is this mean the windows platform don't need the function such as share memory(CreateFileMappingW,MapViewOfFile) inside the OS_WINCE swither? The shared memory is needed on WinCE to implement the LockFile functions (which don't exist on WinCE). If you don't need to do locking (because only your app will access the db), you don't need the shared memory thing. whether I should realize the share memory lock function in the embeded os? I have find there is not a direct way similar to the windows share memory and the interface funcitons. It seems difficult to simulate the share memory funciton and it's lock function in my embeded os. Does it mean I must realize it. or the porting will fail. It's up to you to know what level of compatibility you need. I would guess that for you embedded os you don't need the locking part, so can safely replace it with dummy functions that always succeed. another question: There is also a little difficult to realize the sqlite3WinThreadSpecificData function to get the thread information, Is this also must realize ? If you use threads, then that would depend on your use of sqlite. Regards, ~Nuno Lucas thanks a lot. allen.zhang - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: How to sort not binary?
On 5/13/07, Yves Goergen <[EMAIL PROTECTED]> wrote: On 12.05.2007 22:57 CE(S)T, Ingo Koch wrote: > Fortunately your guess is wrong. ;-) System.Data.SQLite supports > user defined collation sequences. See TestCases.cs of the source > distribution for samples how to implement them. Ah, now I realised that I'd also like to have that "natural sorting", meaning this: ... Can this be done with a user-defined collation, too? How does this all impact on the performance? I have over 5.000 rows to sort on 3 columns and would like to do that in near-real-time. It's up to you to feed with a collation that does natural sort. A few thousand rows are not much in modern computer terms, but I have no idea on the impact in terms of performance the .NET wrapper has. Test it and you'll know. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] file structure issue
On 5/23/07, weiyang wang <[EMAIL PROTECTED]> wrote: does anyone know how can i get the source codes with the early file structure? (65 seperate files)? thanks in advance. Download the complete source code [1] and run "./configure" on it (you need a POSIX build system, like linux, cygwin or MSYS). After that is done, you should have all preprocessed files generated, so you can just copy them to where you want. Regards, ~Nuno Lucas [1] http://www.sqlite.org/sqlite-3.3.17.tar.gz wang - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQL error: disk I/O error
On 5/23/07, Shilpa Sheoran <[EMAIL PROTECTED]> wrote:
Linux 2.6.9
and the media is Nand Flash memory.
/dir1/dir2/dir3
/dir1/dir2 is readonly (cramfs)
dir3 is read write (Flash mem). and I'm creating the database in dir3.
There is your problem. The file system on that directory (I would
guess JFS2 or similar), doesn't support the fsync() call.
The reason has probably to do with the fact the system driver reserves
the right to choose when to do the real write (else a bugged program
or even a malicious user could wear off the device).
That is probably done every 5 seconds or so (is usually configurable
by a kernel parameter), but has the drawback of messing with the ACID
nature of SQLite.
Regards,
~Nuno Lucas
Somehow I don't have a problem in a tmpfs.
The strace showed no diff between tmpfs and this directory where it is
giving I/O error.
Thanks
On 5/22/07, Joe Wilson <[EMAIL PROTECTED]> wrote:
> What is the OS you're using and what kind of media is it?
> Hard drive or USB key or ???
>
> --- Shilpa Sheoran <[EMAIL PROTECTED]> wrote:
> > It seems that rc = fsync(fd); is failing in function
> > static int full_fsync(int fd, int fullSync, int dataOnly) in file os_unix.c
> > {
> > #else /* if !defined(F_FULLSYNC) */
> > if( dataOnly ){
> > rc = fdatasync(fd);
> > }else{
> >
> > //*this call is failing
> >rc = fsync(fd);
> > }
> >
> > }
> >
> > using -DSQLITE_NO_SYNC in the Makefile works
> > What is the way out for this problem?
> > What happens if we use this option -DSQLITE_NO_SYNC ?
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] sqlite internal structs don't make use of C bitfields?
On 5/31/07, Eduardo Morras <[EMAIL PROTECTED]> wrote:
At 23:25 30/05/2007, you wrote:
>Setting and reading individual bytes (u8 in sqlite-speak) are not
>threadsafe either. Only reading/setting entire entire words
>are threadsafe on most architectures.
Using a uint32 for store the flags is threadsafe. There are less than 32
true/false values and read/set is simple. I see no difference doing
if (uint8==0){ // read/test bit
uint8=1; // set bit
whatever more
}
Not atomic, so not thread-safe.
You have a race condition waiting to happen.
and
if (uint32&){ // read/test bit
uint32&&=MASK;// set bit
whatever
}
Also not atomic, so not thread-safe.
in speed, and a compiler should not make worse code on last one. So say
>> Also, my take on bitfields is that they are not thread/multi processor
friendly (there is no
>> atomic "set bit"), and also compilers typically don't optimize well with
that (so before
>> applying this patch, I would test on other platforms than gcc linux x86).
is not true.
It's true not all CPUs have an atomic "set bit" operation.
Regards,
~Nuno Lucas
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] sqlite3_open() fails on WinCE due to utf8ToUnicode / unicodeToUtf8
On 6/1/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Hello It looks like there's a problem on WinCE operating system with the utf8ToUnicode() / unicodeToUtf8() conversion functions. No database can be opened by sqlite3_open() because thehes functions fail. Please have a look what happens there (location: os_win.c :) [...] If I use the CP_ACP flag, everthing works ok: nChar = MultiByteToWideChar(CP_ACP, 0, zFilename, -1, NULL, 0); Same failure probably in unicodeToUtf8() because it uses "WideCharToMultiByte(CP_UTF8, 0, zWideFilename, -1, 0, 0, 0, 0);" My question: Is this a bug or is something wrong with my WinCE-Image? It probably means your WinCE OS image doesn't have UTF-8 support built-in, so you either have to make your own character set conversions or rebuild the OS image with that support (if you can change the image, that is). Regards, ~Nuno Lucas Version information SQLite 3.3.17 (also tried 3.3.13 and failed) WinCE 5.0 Regards Daniel - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: Re: [sqlite] sqlite3_open() fails on WinCE due to utf8ToUnicode / unicodeToU
On 6/1/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Yes, I can rebuild the image, but i didn't find any UTF-8 code page. But it really looks like this code page is not installed, because I tried GetCPInfo(CP_ACP, ) and it failed with ERROR_INVALID_PARAMETER. Can you tell me where I can enable UTF-8 support. (hope this is not to offtopic). Sorry but never used the WinCE builder tool. Maybe someone here knows better. Regards, ~Nuno Lucas Regards, Daniel - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] sqlite internal structs don't make use of C bitfields?
On 6/2/07, Eduardo Morras <[EMAIL PROTECTED]> wrote:
At 17:18 31/05/2007, you wrote:
>On 5/31/07, Eduardo Morras <[EMAIL PROTECTED]> wrote:
>>At 23:25 30/05/2007, you wrote:
>>>Setting and reading individual bytes (u8 in sqlite-speak) are not
>>>threadsafe either. Only reading/setting entire entire words
>>>are threadsafe on most architectures.
>>
>>Using a uint32 for store the flags is threadsafe. There are less than 32
true/false values and read/set is simple. I see no difference doing
>>
>>if (uint8==0){ // read/test bit
>>uint8=1; // set bit
>>whatever more
>>}
>
>Not atomic, so not thread-safe.
>You have a race condition waiting to happen.
>
>>and
>>
>>if (uint32&){ // read/test bit
>>uint32&&=MASK;// set bit
>>whatever
>>}
>
>Also not atomic, so not thread-safe.
I think i have not explained too well. I have wanted to point that if you can
modify/set 8 or 9 values atomically/thread-safe you can do it with one 32 bit
flag value.
Ok, but you can't do it in a portable way without some help (e.g. some
threads library, like pthreads, or the use of the Interlocked*
functions on Windows).
You have instructions on the x86 that you can use for this, but some
CPUs don't (many RISC ones), so you need to have some kind of memory
barrier until you complete the operation.
There are other things you may want to look at. For example, on the
x86, reading a 32 bit value from memory is only atomic if the address
is aligned, else you are actually doing 2 memory reads (meaning
another thread can change the value halfway). Other CPUs fix this by
simply not allowing un-aligned memory reads (causing a lot of problems
when porting x86 code to other platforms).
The subject is a bit more complex than this, because to really talk
about it we would also need to talk about what the C standard says
about the volatile keyword and how different compilers treat it. But
it's becoming off-topic.
Regards,
~Nuno Lucas
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] ALTER TABLE ADD COLUMN - strange behaviour....Mac OSX
On 6/3/07, Mark Gilbert <[EMAIL PROTECTED]> wrote: Anyone have *any* idea what is happening ? I don't know nothing about MacOS, but you may want to check the result of sqlite3_close. It's possible it's not closing the database [1]. Regards, ~Nuno Lucas [1] http://www.sqlite.org/capi3ref.html#sqlite3_close Cheers mark - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] No space left on device?
On 6/4/07, Alberto Simões <[EMAIL PROTECTED]> wrote: What happens when no space is left on device? Does the process die, or does it try to store as much as it can using main memory? SQLite should return an error indicating it failed to write to disk and the reason indicates there is no space left. It's up to your application to handle the error gracefully. I'm asking this because I have a process dying (being killed) because it exauted main memory. That is probably a memory leak in you application (maybe some bug in the error path). Regards. ~Nuno Lucas Cheers Alberto -- Alberto Simões - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Stack usage
On 05 Jun 2007 13:35:33 +0200, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I'v read in change log that some stack allocted memory were moved to the heap, but I think that there is still to much allocated memory on the stack. After creating a table with 2000 columns, jdbc driver created a query that run out of stack. Default java's stack limit is low, but it wasn't hard to create simillar query that crashed C application with default stack limit. And the fact that it crashed instead repoting an error isn't really nice. The query created by the driver looks like that: select null as TABLE_CAT, null as TABLE_SCHEM, 'test' as TABLE_NAME, cn as COLUMN_NAME, dt as DATA_TYPE, tn as TYPE_NAME, 20 as COLUMN_SIZE, 20 as BUFFER_LENGTH, 10 as DECIMAL_DIGITS, 10 as NUM_PREC_RADIX, colnullable as NULLABLE, null as REMARKS, null as COLUMN_DEF, 0as SQL_DATA_TYPE, 0as SQL_DATETIME_SUB, 20 as CHAR_OCTET_LENGTH, ordpos as ORDINAL_POSITION, (case colnullable when 0 then 'N' when 1 then 'Y' else '' end)as IS_NULLABLE, null as SCOPE_CATLOG, null as SCOPE_SCHEMA, null as SCOPE_TABLE, null as SOURCE_DATA_TYPE from (select 0 as ordpos, 1 as colnullable, 'col1' as cn, 'double' as tn, 8 as dt union all select 1 as ordpos, 1 as colnullable, 'col2' as cn, 'double' as tn, 8 as dt union all select 2 as ordpos, 1 as colnullable, 'col3' as cn, 'double' as tn, 8 as dt union all select 3 as ordpos, 1 as colnullable, 'col4' as cn, 'double' as tn, 8 as dt union all select 4 as ordpos, 1 as colnullable, 'lastcol' as cn, 'double' as tn, 8 as dt); but uses more columns. Are you sure it is sqlite that used the stack and not the jdbc driver (or your application)? What happens if you run that query from the sqlite shell? The reason I ask is because I used sqlite on WinCE (but not currently) and only 1 time I had problem with stack usage (a version that made sqlite use the stack instead of the heap, but got corrected right away in the next version). I'm sure a lot of sqlite users in the embedded field would be "ranting" about sqlite stack usage if that was true. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: Re: [sqlite] Stack usage
On 05 Jun 2007 14:59:40 +0200, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Are you sure it is sqlite that used the stack and not the jdbc driver > (or your application)? yes > What happens if you run that query from the sqlite shell? That query I pasted works. Bigger doesn't. Ok, I confused 2000 columns with 2000 rows. With that many columns I'm not surprised, but also think you "get what you deserve". It seems the stack is used in a critical path, I mean, not using the stack would hurt performance (in terms of cpu cache) in the general case so, in my opinion, your case doesn't "deserve" to be fixed. You can make the generator create a temporary table, insert the data on it, make the select and then drop the table, even if that would involve more coding (at least to handle the final table drop after geting the result). Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Stack usage
On 05 Jun 2007 16:40:32 +0200, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Joe Wilson napisał(a): > At least theres a known workaround, so no problem. Workaround is not a solution. For an embedded (and lite) SQL engine like SQLite, you have to bear in mind some features will never be implemented, but you can always open a ticket and "lobby" for that feature to be implemented. > You should ask for your money back. Yes it's free. But I think that no user (or author) of software like when software crashes. Unfortunatelly SQLite does. I would agree with you that it is a bug if the the library crashes, but I don't think there is any portable way for sqlite to know the stack size (or available stack bytes), so it's that kind of error SQLite simply can't do nothing about. To tell the truth, I don't know of any easy non-portable way of doing it either in Windows or Linux. You should file a bug report, anyway, because the developers know better than me. If for nothing else, it could be used to update the documentation about unions to warn about this case. Last time wasn't sent to the mailing list so once more: Does anyone know other places apart from unions where deep recursion may be used? I believe your question is more for other places where you can avoid the deep recursion (as the deep recursion will always lead to the same problem). Best regards, ~Nuno Lucas -- Kasia Cichopek eksponuje biust >>> http://link.interia.pl/f1a6f
Re: [sqlite] sqlite3_create_function function name when called?
On 6/12/07, Omar Eljumaily <[EMAIL PROTECTED]> wrote:
When you create a function with sqlite3_create_function, the callback
function is something like:
myFunc(sqlite3_context *context, int argc, sqlite3_value **argv)
{
}
Is it possible to get the name of the function that the callback was
called because of? I want to write just one callback that is used for
multiple functions. The reason I want to do this is to be able to add
functions at runtime.
I don't think you can obtain the name from your function, but maybe
you can do what you want with the sqlite3_set_authorizer() method
(maybe in conjunction with the experimental sqlite3_overload_function
method to assure an empty function exists).
Regards,
~Nuno Lucas
Thanks.
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: Re: [sqlite] sqlite3_open() fails on WinCE due to utf8ToUnicode / unicode
On 6/7/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: It really looks like this UTF-8 codepage is not avaiable. Is there any WinCE developer that uses SQLite newer than version 3.3.9 on this list? -> Did you have similiar problems since the unicode conversion functions were changed? I haven't used sqlite in WinCE for some time, and haven't answered before because I was waiting to test this myself (which I haven't done yet). I believe this was put because UTF-8 is supported by default in WinCE 5.0 (at least that was my impression), and this is the right fix, but maybe the code should have a fallback to the earlier behaviour if CP_UTF8 is not supported on the device (as it happens with most WinCE 4.x and older devices). Did you open a ticket for this? Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Help with compiling 3.3.17 version for WinCE
On 6/16/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I've download the amalgamation (single .c file) version of SQLite 3.3.17 and I'm trying to compile it using Embedded Visual C++ 3.0, but I'm getting some compiling errors such as,fatal error C1083: Cannot open include file: 'assert.h': No such file or directoryIs there any special settings I need to make to compile the amalgamation version of the code?Thanks for your help.Dave You are compiling it using an old SDK. Only the SDK for WinCE 4.0 and above include "assert.h". If you want to use sqlite on WinCE 3.0, you may want to look at the old sqlite-wince.sf.net code, which include compatibility headers for assert.h and time.h (and support for WinCE 2.x using the legacy 2.8.x branch). Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Unicode collation
On 6/27/07, Trevor Talbot <[EMAIL PROTECTED]> wrote: On 6/25/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > > > So the choices seem to be: > > > > > > (1) Databases that corrupt if you move across platforms. > > > (2) A 10MB database engine > > > (3) Leave things as they are > OK. Here is a crazy idea for consideration: > > You know that you can create a custom collating function as a > DLL or shared library and load it at runtime, right? This has > been a capability since version 3.3.7. Suppose we define a > special table in the database file that is designed to hold > DLLs and/or shared libraries. After thinking a bit, it occurs to me that there's a compromise for the Unicode case that might be workable. The algorithm for collation is pretty stable, it's just the locale data that's the problem. If SQLite understands the algorithm, then locale data can go into special tables in the database itself. Applications manipulating the database schema would need to have the relevant collation data on hand to fill in the database, but other apps concerned with only the data could operate without any special knowledge. This approach keeps the database internally self-consistent while avoiding platform and versioning issues. It's a thought. I love your idea and seems the most promising by far. If we could have it in a way such that people who only use a specific set of locales (like applications that are only internationalized for the german, spanish and italian market, for example) could only have the locale data they need (like ignore hebrew or chinese), I believe that would be the best of all worlds. Maybe we could have a database (the one with the 10MB in size) in the contrib area with all the locale data and people would then import the locales they need into their own databases on creation. Indexes would continue to be coherent across platforms and different locales (with the proper care). This is good even for embedded devices, as they are usually deployed on a single locale, and can have only the locale data they need. Best regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Unicode collation
On 6/27/07, Jiri Hajek <[EMAIL PROTECTED]> wrote: Actually, reading one of the links you posted (http://blogs.msdn.com/michkap/archive/2005/05/04/414520.aspx - everybody please read it before continuing in discussion), I got (what I think is a great) idea: Let's include version information about collation algorithm we are using. So, extending my previous suggestion, the SQLite collation title would look like 'en_AU_Win_2_1', where the individual parts mean: 'en_AU' - obviously a sort order 'Win' - that we are basing the ordering on default Windows methods (otherwise could be e.g. 'ICU') '2_1' - version string as retrieved from GetNLSVersion() function and NLSVERSIONINFO struct (see the link above for details). I don't see how that is different than the previous difficulty of having incompatible databases between systems. The only new thing is we can check it, but the database will have inconsistent behaviour anyway. This way we are absolutely sure that we don't corrupt SQLite database. While I'm working on a single system, the version string wouldn't change. Then, e.g. if I open SQLite database with 'en_AU_Win_2_1' collation on a system NLSVERSIONINFO returns version 2.3, I know I have to reindex, change collation title to 'en_AU_Win_2_3' and then can safely use the database. Trevor proposed method doesn't need that, because the collation data goes with the database (which is the big advantage to me). And if you want to use another collation, you only need to import the collation data needed for that locale. It seems to me to be a really cross-platform solution and what's best - without any coding on SQLite side necessary (even though possible to handle some parts of this proposal internally). On the articles you have pointed (but I don't have a link right now) also states that SQL Server and the Jet engine don't use the system collation functions, but instead have their own integrated collation system (based on the Windows one, off course) exactly for the same reason. For small databases, the task of re-indexing is not big, but you are forgetting you can have a database in a shared network folder, used by PC's in different parts of the world and even different OSs (with samba/cifs). That's why I like Trevor's idea so much. Best regards, ~Nuno Lucas Jiri - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Unicode collation
On 6/27/07, Jiri Hajek <[EMAIL PROTECTED]> wrote: > Trevor proposed method doesn't need that, because the collation data > goes with the database (which is the big advantage to me). And if you > want to use another collation, you only need to import the collation > data needed for that locale. Well, Trevor's proposal sounds good indeed, but realistically, I don't see any big chances it to be implemented - collations aren't that simple and maintaining all the related issues without relying on some external code is probably too much to expect. Well, MySQL does it since v4.1, and from what I googled (I don't have the links on this PC), the collation data needed with the adjustments (tailoring) for the different locales [1] is not that much (and it doesn't change the algorithm code, only data tables are needed). From what I have seen, the Unicode Collation Algorithm (UCA) [2] is already a good base for general ordering, and it seems the European Ordering Rules (EOR) [3] is similar to UCA with the Default Unicode Collation Element Table - DUCET, which already is good enough for many people (including 90% of my needs). Yesterday it looked like unicode.org was down and I haven't had the chance to look at the UCA report (UTS #10) in depth, but I will try to come up with a proposal for sqlite. My idea is to implement the UCA collation in SQLite (with the usual OMIT_* #ifdef's), using the DUCET table as base, and if people need the tailoring part for localized sorting, have it be optional by having a "sqlite_collation_data" table with the needed locale data included on the database. One thing I noticed is that "collations" != "case change". This will not make it possible to use UPPER/LOWER with the same data on the table, but maybe we can work on something in that respect, also. Regards, ~Nuno Lucas [1] http://developer.mimer.com/collations/charts/index.tml [2] http://www.unicode.org/reports/tr10/ [3] http://en.wikipedia.org/wiki/European_Ordering_Rules Jiri - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Unicode collation
On 6/28/07, Jiri Hajek <[EMAIL PROTECTED]> wrote: > My idea is to implement the UCA collation in SQLite (with the usual > OMIT_* #ifdef's), using the DUCET table as base, and if people need > the tailoring part for localized sorting, have it be optional by > having a "sqlite_collation_data" table with the needed locale data > included on the database. That would certainly be great if this is implemented. Note, however, that it doesn't fully solve the issues described in this thread - i.e. if you create a DB by some SQLite version and then use it by a newer version, where some elements were added/modified in DUCET, indexes of these DBs wouldn't be compatible. It can be resolved in several ways, e.g. as suggested to have all DUCET data stored in a special table in SQLite database. It's just a matter of choosing a well-balanced solution... There are 2 problems: UCA changes and DUCET (and/or other locale data) changes. DUCET and locale data is in the database table, so it can only change by user intervention, meaning it's his fault if done without rebuilding the affected table index(es). UCA changes are more problematic, but they are less frequent (it seems there was a minor change between Unicode 4.0 and 4.1, though). I don't see any good solution for this other than having an extra field on the database file (or in the collation data tables) with the UCA version and advise the user when using an UCA version different from the one it was created (or last used, as by default no collation data is needed). It's very probable that not many users are actually bothered by this (the algorithm is probably not changing much over time, and probably not in incompatible ways for most locales). Maybe we can just make sure "PRAGMA consistency_check" notices if there is an inconsistency and that VACUUM will fix this. Btw, even if this is implemented, there is still a need for a standardization such new collation names. E.g. that new language neutral collation could be called Unicode or DUCET? And how about language specific collations? After some thoughts, I'd suggest something like UNIL_en_AU (where UNIL means Unicode linguistic - i.e. some characters are properly ignored, given for example by an ordering of 'con', 'coop', 'co-op') and UNIS_en_AU (where UNIS means Unicode strings - i.e. special characters aren't ignored, so that above words would be ordered as 'co-op', 'con', 'coop'). I don't find this particularly important, because the collation name has to be on the tables, so it can be called "DEFAULT" and have only data for the "fr_FR" locale on the "default" tables (including the DUCET base embedded) on embedded devices. I'm thinking there will be a "reference" database with all locale data, and it's up to users to use it "as is" or build their own (maybe just rename locales). It seems more natural to use the standard C locale names to me (the usual "pt_PT" and "pt_BR" for Portuguese/Portugal culture and Portuguese/Brazil culture) , but I'm open to suggestions when that problem arise, and I'm sure there are already standards we can follow in relation to that. Well, I will probably only have time to actually put words into code next weekend, so I will say something when I have source code to show. Best regards, ~Nuno Lucas Jiri - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Unicode collation
On 6/28/07, Trevor Talbot <[EMAIL PROTECTED]> wrote: On 6/28/07, Nuno Lucas <[EMAIL PROTECTED]> wrote: > One thing I noticed is that "collations" != "case change". This will > not make it possible to use UPPER/LOWER with the same data on the > table, but maybe we can work on something in that respect, also. If the intent is case-insensitive comparisons a la LIKE, note that "upper" and "lower" operations alone won't suffice, because in some scripts a single character maps to multiple characters in one direction. Unicode defines a "case folding" operation for such uses, which does full expansion no matter which direction it is (lower->upper or upper->lower), allowing the result to be handled with a simple memcmp()-style binary comparison. Right, an example is the german "straße" -> "STRASSE" (the 'ß' - beta - character being replaced by "SS" in uppercase, for those who can't see the mail properly). It should certainly be possible to cover those operations too, it's just as nuanced as the collation work :) Yes. I'll try to think on this too, but not the big priority. Best regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: Unicode collation
On 6/28/07, Trevor Talbot <[EMAIL PROTECTED]> wrote: On 6/28/07, Stephane Bortzmeyer <[EMAIL PROTECTED]> wrote: > > RFCs 4646 and 4647 cover this convention. > > No, "language tags" (specified in RFC 4646) use the dash as a > separator, unlike the POSIX locales. So, it is pt-BR. Whoops, good catch. And in a typical identifier, underscore is valid while dash isn't (due to being an operator). Ah well, as close as we can get that makes sense I suppose. > See http://www.langtag.net/ Nice collection of info, thanks. After reading some more it seems language and country tags alone aren't enough. Inside the same country, the same language and even the same culture, different orderings can be used depending on the ordering objective. As an example, there can be a specific order used in phone lists and other for dictionaries. I will leave the decision of the name convention for the last part of the job. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Question about triggers
On 6/29/07, Kees Nuyt <[EMAIL PROTECTED]> wrote: On Wed, 27 Jun 2007 20:42:18 -0400, you wrote: >1) Faster to do the timestamp in the sql statement, since it does not >require activation of the trigger. >2) Query looks 'cleaner', not having an explicit datetime() inside >3) Can switch the timestamp between UTC and localtime by modifying the >trigger as opposed to changing and recompiling the application. 4) Triggers can't be forgotten, so the stamping will be more consistent, much like constraints. and 5) use "DEFAULT CURRENT_TIMESTAMP" in the create statement. Example: CREATE TABLE x ( a, b DEFAULT CURRENT_TIMESTAMP ); INSERT INTO TABLE x (a) VALUES (12345); b has an automatic timestamp. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: Re: [sqlite] sqlite3_open() fails on WinCE due to utf8ToUnicode / unicode
On 6/12/07, Nuno Lucas <[EMAIL PROTECTED]> wrote: On 6/7/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > It really looks like this UTF-8 codepage is not avaiable. Is there > any WinCE developer that uses SQLite newer than version 3.3.9 on this > list? -> Did you have similiar problems since the unicode conversion > functions were changed? I haven't used sqlite in WinCE for some time, and haven't answered before because I was waiting to test this myself (which I haven't done yet). I believe this was put because UTF-8 is supported by default in WinCE 5.0 (at least that was my impression), and this is the right fix, but maybe the code should have a fallback to the earlier behaviour if CP_UTF8 is not supported on the device (as it happens with most WinCE 4.x and older devices). Did you open a ticket for this? I have created ticket #2479 [1] with a possible solution for this problem. As I currently don't have a Windows machine nor a cross-compiler set up, I would appreciate if people from the windows world could test it. Best regards, ~Nuno Lucas [1] http://www.sqlite.org/cvstrac/tktview?tn=2479 - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Changing Database Encoding
On 7/30/07, Mitchell Vincent <[EMAIL PROTECTED]> wrote: > I've read a few places that it is not possible to change the encoding > of a database once it's created. Is it possible to do it in some > automated way with any of the command line utilities? Read about the "pragma encoding" [1] SQL command (you just need to use it before inserting any data). Note that the "encoding" is always UNICODE, so you don't gain much with this pragma if you're from the "western" part of the world > I converted a database from SQLite 2.X to 3.X using sqlite.exe's .dump > function and apparently didn't set the encoding correctly as any > non-English text isn't accessible. I think I need to set the encoding > to UTF8 now, though it "just worked" before, so I'm not sure what I > accidentally did right the first time :-) I don't think that is your problem. You were probably using a 2.x database in "8-bit" mode, not UTF-8. Things can get ugly if you used the UTF-8 library version on non-UTF-8 strings. The only solution I can see is to use something like "iconv" to translate the dump to UTF-8 before inserting the data into a 3.x database. That can be more difficult than you think in case of mismatched use of library versions. Regards, ~Nuno Lucas > > Any help is appreciated! Thanks! > > -- > - Mitchell Vincent - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Unicode Again... Still Stuck... A Challenge... Store and retrieve the word résumé without using a unicode string literal
First, let me start by saying I don't have much experience with
Python, but this isn't a python problem.
On 7/31/07, wcmadness <[EMAIL PROTECTED]> wrote:
[...]
> I'm on a Windows machine. It turns out that the default code page on
> Windows is cp437. So, in my Python code, if I type:
Wrong. The default code page on windows depends on the Windows
localization version. Your default code page can be CP437 (US ASCII)
on your machine, but can be another completely different on another
machine.
> s = 'résumé' (with the French e s), it is stored as: 'r\x82sum\x82' because
> hex 82 (decimal 130) is the code for French e in code page 437 (used by
> Windows)...
>
> OK. So, now that I now my data comes to me from the HTML form (or in a flat
> file) in code page 437 on a Windows machine, I can do the following when I
> send the data to the database:
Wrong. It's the one who generates the HTML page who decides what
encoding the data is interpreted and passed. The default one acording
to most current standards is UTF-8, but most windows users expect it
to be ISO-8859-1 (Latin 1, which is almost identical to the first 256
characters of Unicode) and don't use the appropriate "" tag to
certify that. This explains most problems with international
characters in IE (which tries to "guess" if not given and usually ends
with ISO-8859-1) and Firefox (which doesn't do the "guessing" part and
stick with the standard).
In reality, Windows doesn't use ISO-8859-1, but a variation of it
(usually WIN-1252). The main difference is that Latin 1 considers the
32 characters after 128 as control characters and Windows use this
zone to include extra characters (as an example, the Euro sign was
added there, because the Unicode character code doesn't fit on 8-bit).
You need to learn how to use the "" pragmas on the HTML pages.
Without it. it's the HTTP server who decides which encoding the pages
are (if using HTTP version 1.1 and higher). which is probably wrong on
most cases.
That is the second option if all your pages are on a specific code
page. Configure the server to return a specific code page by default,
but I don't like that option as the server has no way of knowing if
the content it's serving is in fact the encoding it says it is.
You also seem to be reading data from mails. Those are other standards
which you need to read, but Outlook is famous for not following those
standards, so it means a lot of hacks to to have it right.
This are just notes for you. I'm not even an expert on this.
Regards,
~Nuno Lucas
> f = cgi.FieldStorage()
> cur.execute("insert into test values (?,?)",
> (f['txtName'].value.decode('cp437')))
>
> The decode method after the incoming form data will force a translation from
> code page 437 to unicode (from 1 byte per character according to extended
> ascii set code page 437 to 2 bytes per character -- unicode). That's all
> fine.
>
> Now, when I get the data with:
>
> cur.execute("select * from test")
> mylist = cur.fetchall()
>
> I would expect that I would need to encode the unicode data coming from
> Sqlite to get back to my original code page 437 (of course, I could also
> just use the data as unicode). So, I would expect to do this:
>
> (say that row one, column one has the value of résumé)
>
> In that case, the following should return me exactly to the original
> 'r\x82sum\x82'
>
> mylist[0][0].encode('cp437')
>
> But it doesn't!!! (Wacky)!
>
> Rather, it gives me this: 'r\xe9sum\xe9'
>
> Interestingly, that's almost the same as what I get with a unicode literal.
> In other words, if I write this Python code:
>
> x = u'résumé'
>
> and then type x in the shell to see what it is, I get this:
>
> u'r\xe9sum\xe9'
>
> The only difference is that the latter is unicode and the former
> ('r\xe9sum\xe9') is not.
>
> So, to get back where I started, I do the fetchall and then this wacky
> thing:
>
> eval("u'" + mylist[0][0].encode('cp437') + "'").encode('cp437')
>
> In other words, I say: OK, you're almost there. Now, convert to unicode by
> evaluating the string as a unicode literal and then encode the unicode back
> to the code page 437.
>
> What a kludge. It seems like an awefully lot of work to get back to the
> original data that was stored to the database. And why? Does anyone know
> what's going on here???
>
> Thanks.
>
>
> wcmadness wrote:
> >
> > Surely there is an answer to this question...
> >
> > I'm using Python and PySqlite. I'm trying to store the word résumé to a
> > text field. I'm really doing this as a test to see how to handle
> > diacritical letters, such as umlaut characters (from German) or accented
> > characters (from French). I can pro
[sqlite] [RFC] About the sqlite3 collation needed callbacks: design flaw?
The new experimental sqlite3_collation_needed [1] functions seem ideal to use in a collation library, but I believe they have a design flaw that invalidate their use in the generic case. The use case I see for a user using a 3rd party collation library is something like this: 1. open the database 2. invoke the collation library initialization routine 2.1. the collation library would call sqlite3_collation_needed on the database 3. use the database [steps 1-2 could be in a single function passed to sqlite3_auto_extension()] The problem is that we can't actually do this because the collation library has no way to know if the user has already defined some private collation for it's own use, or even use a second 3rd. party library for another collation functions (for example, one library for western languages and another for eastern languages). And even if we could know the previous callback (so we could call it in our function if we don't know how to handle it), then there is the problem of that function pointer being invalidated because it's in a "DLL" we later unload (which is probably the case in a 3rd. party library). I would propose to make sqlite3_collation_needed use some list of callbacks (a list of functions to search in some established direction) and have some way to remove the callback from the list. To avoid defining another function just to remove the callback, one could add some feature flags to define ordering and have a remove flag there, like: int sqlite3_collation_needed( sqlite3*, void*, callback_type_t, int flags ); #define SQLITE3_COLLATION_WANT_UTF8 0x #define SQLITE3_COLLATION_WANT_UTF16 0x0001 #define SQLITE3_COLLATION_REMOVE0x0002 #define SQLITE3_COLLATION_SEARCH_FIRST 0x0004 #define SQLITE3_COLLATION_SEARCH_LAST 0x0008 [the names and values of the constants are just for illustration purposes] This way you add features (choose to insert at front or back of the search list), fix the issue and even remove one function from the API (the UTF-16 variation). The reason I don't implement the code is because I believe it's trivial enough, and if I did then it would have to wait until the legal bureaucracy of copyright assignment was done before being merged into the tree. Best regards, ~Nuno Lucas [1] http://www.sqlite.org/capi3ref.html#sqlite3_collation_needed - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: Re: Re: Re: Re: Re: Re: How does SQLite choose the index?
On 8/6/07, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: > Yes, sure if it fails you might get wet, but I would try the few planks I > got and see which one fits best and not bother with a mathematical model. > All a bit academic as you don't know the situation > I am dealing with. Even in the simplest situation it pays to have a minimum of context analysis before starting the construct. The obvious solution is probably to just extend planks across the river, but if you stop to think a little you may remember the water level on Winter is much higher than the current levels, meaning a much higher pressure of the water on the bridge (and helps to know if it increases in a logarithmic or linear way). By applying the right engineering techniques you at least know the limits of the wood construction you just built, and maybe manage to solve the problem with less resources by having the bridge built in a special way, like in an U shape to increase strength against the current. Regards, ~Nuno Lucas > > RBS > > > > Absolutely. Big bridge or small bridge, if it fails you fall in the > > water. > > > > It looks as if the bridge in Minneapolis failed because construction > > workers moved tons of repaving material onto part of it and overstressed > > that section. A few calculations could have saved the catastrophe. > > > > I saw an estimate that software errors cost just the US more than $100 > > billion per year. That is equivalent to more than 5% of the entire UK > > GDP. Doesn't it make sense to try to build software which works to > > design rather than trying alternatives until one which does not fail > > eventuates? > > > > Also note what early researchers in proof of software accuracy pointed > > out. Testing only finds bugs, it does not establish the correctness of > > a program. Only an appropriate design methodology can hope to establish > > correct behaviour of the program. > > > > RB Smissaert wrote: > >> Poor comparison in this case. > >> Are you going to make a mathematical model when you got a little stream > >> to > >> cross and you have a few available planks to do it? > >> > >> RBS > >> > >> > >> -Original Message- > >> From: John Stanton [mailto:[EMAIL PROTECTED] > >> Sent: 05 August 2007 16:43 > >> To: sqlite-users@sqlite.org > >> Subject: Re: [sqlite] Re: Re: Re: Re: Re: Re: Re: How does SQLite choose > >> the > >> index? > >> > >> We learn mathematics etc so that we can make numerical models which give > >> us design information. Imagine trying to build every combination of a > >> bridge to settle on a design! > >> > >> Make a mathematical model and get it close to optimal at the first > >> attempt. > >> > >> RB Smissaert wrote: > >> > >>>Yes, I suppose you are right there. > >>>I will see if I can put together a report that runs all possible types > >>> of > >>>queries (sequentially) and then see if I have left anything out that > >>> would > >>>cause problems. > >>> > >>>RBS > >>> > >>> > >>>-Original Message- > >>>From: Gerry Snyder [mailto:[EMAIL PROTECTED] > >>>Sent: 05 August 2007 03:35 > >>>To: sqlite-users@sqlite.org > >>>Subject: Re: [sqlite] Re: Re: Re: Re: Re: Re: Re: How does SQLite choose > >> > >> the > >> > >>>index? > >>> > >>>RB Smissaert wrote: > >>> > >>> > >>>> I think an application that > >>>>would produce all the needed indexes based on the table and all the > >>> > >>>possible > >>> > >>> > >>>>queries would be helpful. Anybody done such an app? > >>> > >>>_All_ possible queries? Not practical for any significant number of > >>>columns. N factorial gets big fast. > >>> > >>>The indexes would be much larger than the data base itself. > >>> > >>>I'm afraid you are going to have to settle for doing an intelligent > >>>design of the data base. > >>> > >>> > >>>Gerry > >>> > >>> > >>> > >> > >> > >> > >>>- > >>>To unsubscribe, send email to [EMAIL PROTECTED] > >>> > >> > >> ---
Re: [sqlite] SELECT INTO ... not supported?
On 8/18/07, John Machin <[EMAIL PROTECTED]> wrote: > Something as simple as > SELECT * INTO tblcopy FROM tbl; > (where tbl is an existing table) gets the following error: > SQL error: near "INTO": syntax error > [version: 3.4.2 on Windows XP] > > I note that this syntax is not mentioned on the supported SQL syntax web > page for SELECT, but it's also not mentioned AFAICT in the unsupported > features list on the wiki. > > Am I missing something simple, or is "SELECT INTO" definitely not supported? > > Can anybody help with a workaround that doesn't need an explict (and > rather long) list of fields in the 3-table join that is my real > non-simple requirement? http://www.sqlite.org/lang_insert.html Regards, ~Nuno Lucas > Thanks in advance, > > John - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Odd error on "BEGIN IMMEDIATE"
On 8/18/07, Scott Derrick <[EMAIL PROTECTED]> wrote: > I have a process that repeatedly, once a second, inserts data into the > database. > > I open the database, on the class instantiation. > > once a second I call a function that; > > exec a "BEGIN IMMEDIATE", with a sleep loop if I can't acquire the > reserved lock. > > Then prepare, step, finalize, exit the function > > When I come back into the function and exec a "BEGIN IMMEDIATE" I get an > error > > "Cannot start a transaction within a transaction". > > Whats wrong? Doesn't sqlite3_finalize(stmt), release the locks, > deletes the prepared statement and causes the database to be updated? No, your assumptions are wrong, that's not it's job. If it did then you couldn't have more than a single statement inside a transaction, don't you think? > Why does the engine think I'm still in a transaction? You didn't end the transaction with either "COMMIT"/"END" or "ROLLBACK". Regards, ~Nuno Lucas > thanks, > > Scott - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Aggregate and query limit
On 8/18/07, Mina R Waheeb <[EMAIL PROTECTED]> wrote: > Hi, > Thanks for reply. I'm still confused about that. I don't know much > about SQLite internals, But what i understanded from your reply that > SQLite fetch any row match the query condation and then apply to it > the GROUP BY then apply to it the ORDER BY is there is any, And when > the LIMIT appears SQLite keep fetching the result from the "start" > until the result set size equals the requested limit. > > Do you mean the LIMIT in SQLite is not optimized? and the performance > of selecting from table contains 1000 rows match the query condation > equal selecting the same condation with limit 990,10? Yes. The limit clause just skips the number of records you want from the result set. I haven't checked the code, but I believe you still gain something because you can also avoid reading full rows if you don't need to (like having an index - or an autogenerated temporary index - you can use) when sqlite does the initial "skip" part. Regards, ~Nuno Lucas > > Thanks, > Mina. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Problem opening a new SQLite3 database file
On 8/23/07, Dennis Achá <[EMAIL PROTECTED]> wrote: > I cannot open a new SQLite3 database file through the command prompt. In > the > windows "run" window, I type "SQLite3 mydatabase.db3" and I get the > following error > message: > ** > "Cannot find the file 'SQLite3' (or one of its components). Make > sure the path and filename are correct and that all required libraries are > available." > *** > The only thing I can do is open the SQLite3.exe file and work directly off > of it, but I > cannot save anything. > > I have the following files in a folder located on the C drive: > > 1. sqlite3.exeapplication > 2. sqlite3_analyzer application > 3. sqlite3.dll > 4. tclsqlite3.dll > 5. sqlite3.def > 6. fts2.def > 7. fts2.dll > > This happens on my two (Win XP, Win 98SE) home computers and my work > computer (Win 2000). Can anybody help me figure this out so I can start > creating my database? I'd > greatly appreciate it. Thanks My advice to you, as not many people still know how to work well with the windows command line, is to get and use one of the sqlite GUIs that let you use sqlite databases using a graphical manager. Just select one in the http://www.sqlite.org/cvstrac/wiki?p=ManagementTools page, preferably an open source one (I'm sure there are good ones that deserve the money one pays for them, but it's stupid to pay before you know the limitations of the free alternatives). The reason I don't explain why your way doesn't work is because if you don't know what a PATH is then it's hard to say how to open a command line and start typing DOS commands. Regards, ~Nuno Lucas > Dennis Achá - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: select round(98926650.50001, 1) ?
On 9/3/07, Serena Lien <[EMAIL PROTECTED]> wrote: > Okay, but even if the rounded result does not have a finite binary > representation, shouldn't the displayed (human readable) representation be > at least truncated to the number of decimal places that were requested in > the round function? Not that I am confusing round with truncate, but surely > it is a more acceptable result? One way you could do this would be to implement your own round() function to return a string instead of a number. That way it would display the right result. Look at func.c [1] to see how round() is implemented. Regards, ~Nuno Lucas [1] http://www.sqlite.org/cvstrac/fileview?f=sqlite/src/func.c=1.174 > > thanks, Serena. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: select round(98926650.50001, 1) ?
On 9/3/07, Doug Currie <[EMAIL PROTECTED]> wrote: > On Monday, September 03, 2007 Arjen Markus wrote: > > > Serena Lien wrote: > > >>round(98926650.5, 1) -> 98926650.501 > >>round(85227887.01, 1) -> 85227887.001 > > > They are in fact rounded, but the internal binary representation can > > not be turned into the appropriate decimal (and human readable) > > representation due to the finite precision. > > 98926650.5 is represented exactly in IEEE double; something else is > mucking up the round or the display of the result, maybe both. > > 85227887.0 is represented exactly in IEEE double, though 85227887.01 > is not. Nevertheless, there are ways to print floating point numbers > readably. http://portal.acm.org/citation.cfm?id=93559 This made me to remember there was a bug some time ago about the rounding algorithm (but can't remember at what version it was fixed), so I just tested it. "official" amalgamated sqlite 3.4.0 downloaded from the site some time ago: SQLite version 3.4.0 Enter ".help" for instructions sqlite> select round(98926650.5, 1) ; 98926650.5 sqlite> select round(85227887.01, 1) ; 85227887.0 sqlite> select round(85227887.01, 2) ; 85227887.01 sqlite> select round(98926650.50001, 1) ; 98926650.5 Linux [K]Ubuntu 7.04 (feisty) sqlite3 package 3.3.13-0ubuntu1: SQLite version 3.3.13 Enter ".help" for instructions sqlite> select round(98926650.5, 1); 98926650.5 sqlite> select round(85227887.01, 1); 85227887.0 sqlite> select round(85227887.01, 2); 85227887.01 sqlite> select round(98926650.50001, 1) ; 98926650.5 $ uname -a Linux ubuno 2.6.20-16-generic #2 SMP Thu Jun 7 20:19:32 UTC 2007 i686 GNU/Linux So it seems SQLite is already doing the right job. Maybe some OS specific error? Wasn't there some discussion earlier about the Microsoft compiler not using the full double precision by default? Regards, ~Nuno Lucas > e > > -- > Doug Currie > Londonderry, NH, USA - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Re: select round(98926650.50001, 1) ?
On 9/5/07, Simon Davies <[EMAIL PROTECTED]> wrote:
[...]
> in sqlite3VdbeMemStringify.
> This resolves down to calling a function vxprintf, which has the following
> line:
>
>while( realvalue>=10.0 && exp<=350 ){ realvalue *= 0.1; exp++; }
>
[...]
> Interestingly, if the line is changed to
>while( realvalue>=10.0 && exp<=350 ){ realvalue /= 10; exp++; }
> then the accumulated error is less, and the correct value is printed.
> Of course, this change may break other builds.
>
> One wonders why XP/VC++ produces such a large error in its floating
> point manipulations that appears to be avoided by other compilers.
What about defining __STD_IEC_559 before the compilation?
Acording to this:
http://david.tribble.com/text/cdiffs.htm#C99-iec60559
C99 allows one to force the compiler to follow the standard, so maybe
libc does it by default, but the official sqlite compiled version
(which IIRC is linked with the old Microsoft C runtime DLL) doesn't.
Regards,
~Nuno Lucas
>
> Rgds,
> Simon
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Re: select round(98926650.50001, 1) ?
On 9/5/07, Cory Nelson <[EMAIL PROTECTED]> wrote: > On 9/5/07, Nuno Lucas <[EMAIL PROTECTED]> wrote: > > What about defining __STD_IEC_559 before the compilation? > > Acording to this: > > > >http://david.tribble.com/text/cdiffs.htm#C99-iec60559 > > > > C99 allows one to force the compiler to follow the standard, so maybe > > libc does it by default, but the official sqlite compiled version > > (which IIRC is linked with the old Microsoft C runtime DLL) doesn't. > > it is defined by the compiler to indicate that it conforms. it is not > something that you yourself define. ugh! Don't know how I got it wrong! Seems like I read it in a wishful thinking way. Sorry for the noise. Regards, ~Nuno Lucas > -- > Cory Nelson - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLite or MS Access
On 9/7/07, Andre du Plessis <[EMAIL PROTECTED]> wrote: [..] > What I don't understand is how Access, and other DB's are able to still > operate much faster (maybe not as fast), and still flush file buffers to > disk, is beyond me. Maybe it really still caches it, I would not be > surprised if you pull the plug from an MS access db you may end up with > missing records even if you committed, I could be wrong... MS Access has several modes of operation, but the default one is not ACID, so it seems fast. There are ways to set the transaction mode to be ACID (which I don't recall what/how they are), which will make the performance suffer behind any hope, making the comparison more fair. I don't recall now, but I seem to remember the default mode doesn't even guarantee the data is on disc when finished using it, which makes inserting a single record an order of magnitude slower on SQLite (because it waits for the data to get to the disk controller) than for Access (which just gives the data to the OS, not caring if it goes to disk or not). In a nutshell, benchmarks are not easy... Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] extension-functions.tgz for sqlite3 3.4.1 ?
On 9/10/07, Joe Wilson <[EMAIL PROTECTED]> wrote: > --- Liam Healy <[EMAIL PROTECTED]> wrote: > > Thanks for the tip Joe. With sqlite3.h included, I can eliminate os.h and > > vdbeInt.h, but not sqliteInt.h. Apparently sqliteInt.h is not included by > > sqlite3.h, and there are typedefs there that are needed. > > It would be nice if people could use the extension functions with > the amalgamation, consisting of just sqlite3.h and sqlite3.c. > If you use sqliteInt.h, they can't do that. > > Why not just repeat the typdefs for u8, etc, in your module? Don't know current compiler standard compliance, but maybe including the "new" header file and using uint8_t, uint16_t, etc. could be better yet (instead of every library having it's own typedef section for basic types). Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLite and Large Databases
On Nov 8, 2007 5:04 PM, PokerAce <[EMAIL PROTECTED]> wrote: > I am using Windows Task Manager to determine memory usage. I minimize my > application (which causes the task manager to show a more accurate > measurement of memory used) and still the memory usage is enormous. Windows Task Manager doesn't count DLL's memory usage twice, it just add that memory to the first linked program, while a second one will appear to consume a lot less memory (but inverting the situation when the order of the programs execution changes). For example, the first .NET application will seem to use a LOT of memory (because all the DLLs it will link to), while the second one may use just a few mega (which is probably the actual memory it's allocating more the program executable size, adding any DLL loaded specific to the program). As other said, use ProcessExplorer (www.sysinternals.com) if you really want to have a meaningful clue on the memory usage of your program. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Problem creating extension for use with load_extension
On Nov 9, 2007 10:19 PM, Bob Dankert <[EMAIL PROTECTED]> wrote:
> First off, I would like to say that although I have a lot of experience
> with programming, most of it is in C#/Java and I do not have a lot of
> experience with C++, although I have been working with SQLite for years.
> I am attempting to create my own extension to use with SQLite but am
> having problems. Using the command line interface, when I load the
> extension I get the following:
>
> SQLite version 3.5.2
> Enter ".help" for instructions
> sqlite> select load_extension('mydblib.dll');
> SQL error: The specified procedure could not be found.
>
Seems like you didn't enable the extension loading mechanism. It
defaults to disabled for security reasons.
Check the wiki page about the SQLITE_OMIT_LOAD_EXTENSION define:
* http://www.sqlite.org/cvstrac/wiki?p=LoadableExtensions
Regards,
~Nuno Lucas
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Performance tuning using PRAGMA, other methods
On 11/20/07, Scott Krig <[EMAIL PROTECTED]> wrote: > There are apparently no folks with the experience to answer the > questions as given? Those who have experience know better than to try to answer in an e-mail what 1000 mails in the mailing list are not enough. The wiki and documentation have more than enough information for the level of detail you want. Regards, ~Nuno Lucas > -Original Message- > From: Tom Briggs [mailto:[EMAIL PROTECTED] > Sent: Tuesday, November 20, 2007 1:41 PM > To: sqlite-users@sqlite.org > Subject: RE: [sqlite] Performance tuning using PRAGMA, other methods > > >re: Q1 - Maybe (hopefully) someone will prove me wrong, but I think > that your question is too broad to be answerable (unless you're actually > attempting to assemble a collection of optimal values in all possible > situations for all existing pragmas... ;shrug) > >re: Q2 - At the risk of sounding crass, tuning queries is, has been > and always will be the best way to optimize the performance of any > database. I've done a lot of tuning of SQLite and a half dozen other > databases, and query design is always what has the most impact. > Pragmas, #defines, API usage, etc. are always a distant second in the > race for performance gains. > >-T > > > -Original Message- > > From: Scott Krig [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, November 20, 2007 4:24 PM > > To: sqlite-users@sqlite.org > > Subject: RE: [sqlite] Performance tuning using PRAGMA, other methods > > > > To the point, the questions are: > > > > > > Q1)) PRAGMA: Does anyone have experience and good results optimizing > > sqlite performance using PRAGMA's? If so, which ones, how were they > > used, and what was the performance increase? > > > > Q2)) Other techniques: Any success stories on sqlite optimization > > methods of any type would be appreciated. > > > > > > > > > > > > -Original Message- > > From: Tom Briggs [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, November 20, 2007 10:40 AM > > To: sqlite-users@sqlite.org > > Subject: RE: [sqlite] Performance tuning using PRAGMA, other methods > > > > > >Which pragmas will be most effective (and what values you > > should use > > for each) depends on what you're trying to do with the database. > > Synchronous is important if you're writing frequently, for > > example, but > > won't matter much in a read-only setting. Appropriate values for the > > page_size and cache_size pragmas vary depending on whether > > the database > > is write-mostly or read-mostly and also depending on whether > > you want to > > optimize for reading or writing. > > > >So in short, the answer is, it depends. Depends on what you're > > trying to tune for, that is. > > > >-T > > > > > -Original Message- > > > From: Scott Krig [mailto:[EMAIL PROTECTED] > > > Sent: Tuesday, November 20, 2007 1:13 PM > > > To: sqlite-users@sqlite.org > > > Subject: [sqlite] Performance tuning using PRAGMA, other methods > > > > > > > > > What are the 'biggest bang for the buck' sqlite optimization > > > techniques > > > to apply to a working system to tune performance? > > > > > > Q1)) PRAGMA: Does anyone have experience and good results optimizing > > > sqlite performance using PRAGMA's? If so, which ones, how were they > > > used, and what was the performance increase? > > > Q2)) Other techniques: Any success stories on sqlite optimization > > > methods of any type would be appreciated. > > > > > > Thanks. > > > > > > > > > Scott > > > -=- > > > > > > > > > > > > Here is a list of the PRAGMA examples from the sqlite documentation: > > > > > > > > > > > > PRAGMA auto_vacuum; > > > PRAGMA auto_vacuum = 0 | none | 1 | full | 2 | incremental; > > > > > > PRAGMA cache_size; > > > PRAGMA cache_size = Number-of-pages; > > > > > > PRAGMA case_sensitive_like; > > > PRAGMA case_sensitive_like = 0 | 1; > > > > > > PRAGMA count_changes; > > > PRAGMA count_changes = 0 | 1; > > > > > > PRAGMA default_cache_size; > > > PRAGMA default_cache_size = Number-of-pages; > > > > > > PRAGMA default_synchronous; > > > > > > PRAGMA empty_result_callbacks; > > > PRAGMA empty_result_callbacks = 0 | 1; > > > > > > PRAGMA encoding; > > > PRAG
Re: [sqlite] Sqlite version for libc 2.1.3
Look in LFS (Linux From Scratch) on how to create a cross-compiler that you can chroot into before the compile. It makes sure only the libraries you installed are the ones your binaries will link to. It should be an easy easy, but quite frankly haven't found it yet (and I am cross-compiling to the x86 platform, but different library versions). Regards, ~Nuno Lucas On 11/23/07, Tara_Nair <[EMAIL PROTECTED]> wrote: > Thanks for your response, Trevor. > > It is what I had initially thought too, that if I built it with an older > set of libraries it will look for those versions at runtime too. But > these libs versions seem to be inbuilt. I cannot seem to change the > sqlite-v3.5.2 dependency on the specific versions of libc-2.2.3 etc. > > I downloaded the code from http://www.sqlite.org/download.html > (sqlite-3.5.2.tar.gz) > I used the following command to configure it : > > $ config_TARGET_CC=arm-linux-gcc config_BUILD_CC=gcc > ../sqlite-3.5.2/configure --disable-tcl --disable-readline --disable-tcl > --disable-threadsafe --host=arm-linux > --prefix=/home/tara/user-driver/sql/install-sq > $ make > $ make install > Then copied the install-sql to my target and try to run it. > > If I forciby link the ld and libc on command line while running sqlite3 > , it works. > That is , if at the target prompt I do the following , sqlite3 works. > > LD_LIBRARY_PATH=/lib/2.2.3:$LD_LIBRARY_PATH > >/lib/2.2.3/ld-2.2.3.so ./sqlite3 > It works for sqlite3 , but the rest of the application dies including > the simple commands like ls, mount etc as the system depends on the > 2.1.3 version of the libraries. > > I am sure this is too much information about my debug. But essentially > my point is if I could get a sqlite3 version which depended on 2.1.3 > version of libraries in /lib folder, it would be great. > > Thanks & Regards, > Tara > > Trevor Talbot wrote: > > On 11/22/07, Tara_Nair <[EMAIL PROTECTED]> wrote: > > > > > >> I just recently started to use Sqlite3 version 3.5.2 on an ARM-LINUX > >> based embedded platform. > >> Sqlite3-v3.5.2. has a dependency on libc, ld and libpthread versions 2.2.3. > >> > > > > Er.. it should depend on whatever versions you link it against when > > you cross-compile for your platform. > > > > How/where did you get the version of SQLite you're trying to use now? > > > > - > > To unsubscribe, send email to [EMAIL PROTECTED] > > - > > > > > > > > > - > To unsubscribe, send email to [EMAIL PROTECTED] > - > > - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] benchmarking UTF8 vs UTF16 encoded databases
On Nov 23, 2007 1:56 PM, Igor Sereda <[EMAIL PROTECTED]> wrote: > > About the endieness, you don't need to know if you > > don't care. SQLite handles it. > > SQLite does handle that, but what would be the performance loss when working > with a UTF-16 encoded database, but with endianness opposite to the system? > That's quite probable scenario, say, a database created on Intel-based > system and then moved to Mac/PPC. If you will be sharing databases between different endienness systems then you care, so you will take appropriate actions to have the best result. The same is true with any other portable file format. Regards, ~Nuno Lucas > > Best regards, > Igor > > > > > -Original Message- > From: Nuno Lucas [mailto:[EMAIL PROTECTED] > Sent: Friday, November 23, 2007 2:01 PM > To: sqlite-users@sqlite.org > Subject: Re: [sqlite] benchmarking UTF8 vs UTF16 encoded databases > > On 11/23/07, Jarl Friis <[EMAIL PROTECTED]> wrote: > > Hi Daniel. > > > > Thanks for the benchmark reports, interesting studies. > > > > Another reason to stay away from utf-16 is that it is not endianess > > neutral. Which raise the question are you storing in UTF-16BE or > > UTF-16LE ? > > If you only speak Japanese and all your characters are 3 bytes or more in > UTF-8 and always 2 bytes in UTF-16 which would you tend to choose? > > About the endieness, you don't need to know if you don't care. SQLite > handles it. > > Regards, > ~Nuno Lucas > > > > > Jarl > > > > - > To unsubscribe, send email to [EMAIL PROTECTED] > > - > > > > - > To unsubscribe, send email to [EMAIL PROTECTED] > - > > - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] benchmarking UTF8 vs UTF16 encoded databases
On 11/23/07, Jarl Friis <[EMAIL PROTECTED]> wrote: > Hi Daniel. > > Thanks for the benchmark reports, interesting studies. > > Another reason to stay away from utf-16 is that it is not endianess > neutral. Which raise the question are you storing in UTF-16BE or > UTF-16LE ? If you only speak Japanese and all your characters are 3 bytes or more in UTF-8 and always 2 bytes in UTF-16 which would you tend to choose? About the endieness, you don't need to know if you don't care. SQLite handles it. Regards, ~Nuno Lucas > > Jarl - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] compilation error in sqlite 3.5.3
On Nov 30, 2007 11:58 AM, Sreedhar.a <[EMAIL PROTECTED]> wrote: > Hi, > > I am trying to compile the latest version of the sqlite 3.5.3 in vc++. > i got the following error. > > Can anyone help me, what i am missing to add. > I just downloaded all the source files and created a project and started > compiling. > > Compiling... > tclsqlite.c > e:\sharing\sqlitedb\sqlite3_5_3\tclsqlite.c(17) : fatal error C1083: Cannot > open include file: 'tcl.h': No such file or directory > Error executing cl.exe. > > Sqlite3_5_3.exe - 1 error(s), 0 warning(s) > > Where i can get this tcl.h file If you don't need TCL support just don't include tclsqlite.c in your project. For you information, TCL is a script language, and SQLite includes "native" support for it. If you don't know about it then I guess you don't need it (it is used to run the test suit, though). Regards, ~Nuno Lucas > > Best Regards, > A.Sreedhar. > > > - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Should the next release be 3.5.4 or 3.6.0?
On Dec 14, 2007 12:35 AM, Steven Fisher <[EMAIL PROTECTED]> wrote: > On 13-Dec-2007, at 8:40 AM, [EMAIL PROTECTED] wrote: > > > My question to the community is this: Are these > > differences sufficient to justify going with version > > 3.6.0 in the next release? Or can we call the change > > a "bug fix" and number the next release 3.5.4? > > I guess I'm in the minority, but I'd find a change in the meaning of > my queries surprising in a bug fix release. That sounds like a 3.6 to > me. Well, I agree with you. It just seems the right thing to do, and version numbers are cheap. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] what platforms does SQLite run on?
On Feb 19, 2008 3:10 PM, Fin Springs <[EMAIL PROTECTED]> wrote: > > Here's a link to an WinCE port: http://sqlite-wince.sourceforge.net/ > Is there a reason to use this port any more? The straight 3.5.6 > amalgamation compiles and runs fine for me on Windows CE 5.0 and > includes specific '#if OS_WINCE' sections for CE (to define localtime, > etc). This port is dead now, because the official sqlite code already handles Windows CE. It can still be usefull if you need to program for old Windows CE systems, meaning older than WinCE 4.0 (i don't recall exactly if in 4.0 itself it compiles out of the box). Other than that, just use the last SQLite version. Regards, ~Nuno Lucas ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
Re: [sqlite] compiling sqlite
On 6/19/06, Robin Cook <[EMAIL PROTECTED]> wrote: Is it possible to compile sqlite on embedded visual c 3.0 for wince 2.11 without MFC and TCL/TK on a Symbol PDT7242 barcode scanner. You should be able to use the 2.8.16 source in the sqlite-wince.sf.net site (maybe with one or other tweak as it's been a while since I last tested it on 2.11). You will not have file locking support, but I guess you will not need it. As for Tcl/Tk, you are on your own, sorry. I have tried with the already preprocessed windows source but keep getting error with missing header files like assert.h etc. The port includes dummy assert.h and time.h files exactly because of this. I am unable to use the dll as it requires msvcrt which is not available on it. You can't use a windows DLL on CE, even if msvcrt was available. You will need to compile your own. Best regards, ~Nuno Lucas Thanks Robin Cook
Re: [sqlite] compiling sqlite
On 6/21/06, Robin Cook <[EMAIL PROTECTED]> wrote: Tried to compile 2.8.16 but got the below errors. Any suggestions? Thanks. C:\Windows CE Tools\wce211\PDT7200\Samples\sqlite\btree_rb.c(314) : warning C4013: 'printf' undefined; assuming extern returning int This is unused debug code. You can #ifdef the whole function. C:\Windows CE Tools\wce211\PDT7200\Samples\sqlite\vdbe.c(389) : warning C4013: 'getc' undefined; assuming extern returning int C:\Windows CE Tools\wce211\PDT7200\Samples\sqlite\vdbe.c(395) : warning C4013: 'ungetc' undefined; assuming extern returning int It seems sqlite is using it's own fgets implementation for the vdbe now, but using C functions not found in the (very poor) 2.11 C runtime. If I remember correctly, you can use fgetc instead of getc, but don't think there is an ungetc replacement, so you'll need to reimplement the function. C:\Windows CE Tools\wce211\PDT7200\Samples\sqlite\vdbe.c(4140) : warning C4013: 'fopen' undefined; assuming extern returning int You'll have to replace this with wfopen (WinCE 2.11 only implements the unicode variant of this functions). Look at wStrDup() in the os.c file for how to transform the path to Unicode. C:\Windows CE Tools\wce211\PDT7200\Samples\sqlite\vdbe.c(4140) : warning C4047: '=' : 'void *' differs in levels of indirection from 'int ' This are the "normal" sqlite warnings ;-) I don't have Windows to test this, but you should now know what to do. Best regards, ~Nuno Lucas
Re: [sqlite] i have a few Qs - sqlite3
Please don't feed the troll... ~Nuno Lucas
Re: [sqlite] Unable to write to database
On 7/12/06, Gregary Hendricks <[EMAIL PROTECTED]> wrote: Greetings fellow SQLiters, I am having a bad hair day as a result of an error I am getting with my SQLite database. I am running SQLite 3.3.6 on SuSE linux 9.3 I am trying to access it via a Perl CGI script on my website. I am using the Perl DBD::SQLite module to access the database. I am able to read from it without any difficulty, the problems arise when I try to do an insert. I get the following error: Errors: 1, unable to open database file(1) at dbdimp.c line 398 The strange thing is that if I run the cgi script from the command line, it works just fine. This seems point to a permissions issue but the file and directories are set to open write (666 and 777 respectively). Any pointers? Maybe sqlite is not able to create the temporary journal file? Try to add perl code to create a file on the same directory to make sure it's writable by the cgi user. Just my .02 cents, as don't know much about perl. Regards, ~Nuno Lucas
Re: [sqlite] How do you find out the names of the fields within a table?
On 7/12/06, John Newby <[EMAIL PROTECTED]> wrote: Hi, how can I find out the names of the fields within a given table? I've tried "pragma table_info(test);" but this brings back too much info, I just require the names as I'll be storing them in an array within my application. When you use pre-compiled SQL statements you get that info for free, that is, whitout actually needing to run the query. Look the sqlite3_column_name function: * http://sqlite.org/capi3ref.html#sqlite3_column_name Regards, ~Nuno Lucas Many thanks John
Re: [sqlite] How do you find out the names of the fields within a table?
On 7/13/06, John Newby <[EMAIL PROTECTED]> wrote: Hi Nuno, I'm using VB.Net 2002 so I don't think I can use these commands. And what about doing a "SELECT * FROM your_table_name LIMIT 0" ? Does your sqlite wrapper returns the column names? If it doesn't, maybe you should think in using another sqlite wrapper that allows it... Regards, ~Nuno Lucas
Re: [sqlite] how to flush database to disk?
On 7/14/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I've just lost a couple of days' worth of data when my app crashed. (Well, the data wasn't a total loss thanks to backup plans, but the database itself essentially reverted to its state of 2 days ago.) This is despite my app doing a COMMIT after every modification of the DB. Any chance of being the OS file system that "rollbacked" the files after the crash? I've seen this before after a Windows CE based machine (with flash memory as disk) crashed. I would assume NTFS on Windows machines can do the same. Regards, ~Nuno Lucas
Re: [sqlite] write/read from BLOB
Just "bind" your data using prepared statements. You can pass null characters inside. Another option is to use the X'0011223344' syntax to insert binary data in hexadecimal format (you can retrieve it in the same format by using the "quote" function). Regards, ~Nuno Lucas On 7/24/06, Kiril Dzolev <[EMAIL PROTECTED]> wrote: Hello, I just started with SQLite couple a days before so I am new and without any experience. My database have only two columns. First column for the Key and other must be a binary data (binary data contains some float or integer numbers). So the question is how to write/read a binary data... I need to read the binary data and change it (write binary data). I am using C++ for executing those commands. Sorry for my English and thanks a lot, Kiril
Re: [sqlite] locked implies exception?
On 7/25/06, Fred Williams <[EMAIL PROTECTED]> wrote: Good Grief! Everybody knows CRUD is what you clean off on US Navy ships at least once a week. Nice to know it only affects US Navy ships ;-) (couldn't resist)
Re: [sqlite] Is there schema caching in sqlite?
On 7/25/06, Ralf Deininger <[EMAIL PROTECTED]> wrote: I have a multi-threaded application. Each thread has its own database connection. One of the thread changes the schema of the database (DROP TABLE, CREATE TABLE). I have observed that the other threads do not know about these modifications since queries for the new/altered table doesn't return the expected results. You are not checking the return codes because you should have received a SQLITE_SCHEMA_CHANGED (or whatever the error code is). If you are not receiving this, then it's either a bug in sqlite or in your code. It looks like the schema is cached by the connection. Can anybody confirm this? How can I efficiently refresh the cache, or do I have to close and reopen the database connection? You need to "re-prepare" all your pre-compiled statements after you receive the error. Maybe you are using some wrapper around the sqlite API that is missing this? Regards, ~Nuno Lucas -Ralf
Re: [sqlite] SQLiteSpy vs. SQLite.dll speed comparison
On 7/22/06, Eduardo <[EMAIL PROTECTED]> wrote: Sorry, but i must disagree. He uses VC6, a compiler from pre-alot-of-processor-advances. So, the compiler can't, not only compile for a more modern and different processor (different pipeline, sse, new processor modes, etc..) but also use old libraries that were made for older processors. For example, in Delphi 6 (if i remember well) code written in pascal run faster than assembly optimized libraries, but written for 386!!!. In powerpc area (what i know), using altivec libraries, boost sort code up to x16, memory2memory copy, assign, move, etc... up to x4 and so on, but if you use a compiler from pre-altivec era or non altivec optimized libraries (including libc), your code will be a lot slower. While I agree with you on the general, I must note that would be for a CPU-bound application. Most sqlite applications tend to be IO-bound (unless you have enough memory and is all cached), so the differences will not manifest so much. As an aside, there are applications that have better performance with -Os (optimized for size) than with any -O, because can incur in dramatically less cache misses (which is several times slower than a cache hit and can also imply scheduling decisions against the process/thread). Regards, ~Nuno Lucas
Re: [sqlite] how to use the sqlite command-line specific commands with the dll version
On 7/26/06, John Newby <[EMAIL PROTECTED]> wrote: Hi, I was wondering how I could use the commands specific to the command-line interface with the dll version. You can't, but see below. The commands I am wanting to use are as follows :- .dump .import .output .read You can look into the shell.c source and just copy the code you want into your own functions. The sqlite shell is all implemented in this file, so it's not too hard to look how it's done there and use in your code. Regards, ~Nuno Lucas Any help would be greatly appreciated Many thanks John
Re: [sqlite] temp_store=1 performance on Mac OS X vs. Windows
On 7/27/06, Joe Wilson <[EMAIL PROTECTED]> wrote: Also, consider running that Task Manager and enable all the column statistics (I/O, threads, VM Faults, Page Faults, etc) to see what's going on while you're running your software. You never know what crazy process might be screwing up your performance. FileMon from sysinternals (bought recently by Microsoft) is very good at showing what processes are doing with what files during some time. Regards, ~Nuno Lucas
Re: [sqlite] Way to "Bind" Columns in Prepare
On 7/30/06, michael cuthbertson <[EMAIL PROTECTED]> wrote: I need to embed variable column names in my queries, such as 'colX > '2004-01-01' where 'X' is a variable. bind_text will not do this. Is there another way? Yes, do another query. ;-) Seriously, note that if you change the condition you will generate a diferent SQL query, meaning it will use different optimization techniques (like using a different index). The bind API is for precompiled SQL statements, where the SQL parsing is done before the actual database access, that is, changing a column name will imply another precompile. Best regards, ~Nuno Lucas Thanks. Michael
Re: [sqlite] RE: UNICODE Support
On 8/4/06, Cory Nelson <[EMAIL PROTECTED]> wrote: I recommend using utf-16 in the database - sqlite doesn't fully support utf-8, and some things may give unexpected results if you use it. Could you give some example of unexpected result with UTF-8? In my experience the only unexpected results with UTF-8 were bugs in my program (like passing non-UTF-8 string). Any other unexpected result should be considered a bug in SQLite and reported as such. Regards, ~Nuno Lucas -- Cory Nelson http://www.int64.org
Re: [sqlite] date data types
On 8/4/06, Mario Frasca <[EMAIL PROTECTED]> wrote: nice. if I was just working inside of SQL and was only interested in printing the values, it would probably be fine. my problem is that I'm co-author of a python library (http://ibo.sourceforge.net) on top of various db-api2 interface libraries to three or four db engines (four, if I can include sqlite). one of the problems consists in writing and retrieving datetime data to the database. the dynamic typing of sqlite is not a problem, actually it fits quite good with the strong dynamic typing system of Python... but then, when I write to just any field a value which is a date, I would like to get back a value which is a date, not a string or a floating point number. the same goes for a datetime and a timedelta, which are each a separate type in python... You need to get the column declared type and convert it to the type you want based on that. would it be difficult, or simply impossible, to implement these types in sqlite? I think the question here is if the developpers want to do it. It wasn't pacific the inclusion of datetime support in sqlite, so adding one more type it's probably a litle more difficult (remember sqlite is an embeded SQL engine, which aims for a small size and low memory footprint). You can always add a ticket for it, as a new feature, and see how it goes ;-) Regards, ~Nuno Lucas
Re: [sqlite] UNICODE Support
On 8/4/06, Cory Nelson <[EMAIL PROTECTED]> wrote: IE, using memcmp() to compare strings. I've been bitten by this before, with SQLite producing unexpected results when using UTF-8. Using UTF-16 has worked more reliably in my experience. SQLite only knows how to sort ASCII, so memcmp does that right (being it UTF-8 or UTF-16). If you think about it, the only way sorting will work 100% is by having some form of localization (because for each language different sorting rules apply, _even_ for words composed only of ASCII characters). Adding localization to SQLite is out of the question (it would probably need a library as big as SQLite itself), so it's up to the user to define it's own localization funtions and integrate them with sqlite (there are all the necessary hooks ready for that). Regards, ~Nuno Lucas
Re: [sqlite] UNICODE Support
On 8/5/06, Cory Nelson <[EMAIL PROTECTED]> wrote: On 8/4/06, Nuno Lucas <[EMAIL PROTECTED]> wrote: > On 8/4/06, Cory Nelson <[EMAIL PROTECTED]> wrote: > > IE, using memcmp() to compare strings. I've been bitten by this > > before, with SQLite producing unexpected results when using UTF-8. > > Using UTF-16 has worked more reliably in my experience. > > SQLite only knows how to sort ASCII, so memcmp does that right (being > it UTF-8 or UTF-16). > > If you think about it, the only way sorting will work 100% is by > having some form of localization (because for each language different > sorting rules apply, _even_ for words composed only of ASCII > characters). > > Adding localization to SQLite is out of the question (it would > probably need a library as big as SQLite itself), so it's up to the > user to define it's own localization funtions and integrate them with > sqlite (there are all the necessary hooks ready for that). I was not talking about sorting in my post - I've had simple = index comparisons fail in UTF-8. You should have reported it. If it's true, it's a bug that needs to be corrected. But again I would say I never found a bug like that in sqlite. But, since you brought it up - I have no expectations of SQLite integrating a full Unicode locale library, however it would be a great improvement if it would respect the current locale and use wcs* functions when available, or at least order by standard Unicode order instead of completely mangling things on UTF-8 codes. For it to respect the current locale then the database would be invalid after moving/using it in another locale (the affected indexes would need to be rebuilt). Using the COLATE thing (which I never used exactly because of the problem above) you can define your own sort function that does what you want. On the second point, you may be right and can be considered a bug. A sorted table should have exactly the same order either if the database is using UTF-8 or UTF-16 internally (even if it doesn't follow the UNICODE order). At least it seems consistency on a query result should be assured on this. Maybe others have another point of view... Regards, ~Nuno Lucas
Re: [sqlite] [ANN]SQLtNoCase - convert SQLite TEXT columns to TEXT COLLATE NOCASE
I don't mind receiving occasional announcements on new programs using SQLite (even if I'll never use them), but don't you think one per day is just too much? Regards, ~Nuno Lucas On 8/8/06, Me <[EMAIL PROTECTED]> wrote: FREE - SQLtNoCase - convert TEXT columns to TEXT COLLATE NOCASE SQLite handles text columns different, depending on how COLLATE is defined when the column is created. The default is COLLATE BINARY. TEXT or TEXT COLLATE BINARY - columns will be treated case sensitive for ORDER BY and WHERE comparisons. TEXT COLLATE NOCASE - columns will be handled case insensitive. 'smith' = 'Smith' For the most part, TEXT COLLATE NOCASE will give the preferred results on short text columns such as [Name]. While SQL can be used to force case insensitive comparisons - converting a column to TEXT COLLATE NOCASE is a permanent solution. Converting a column from TEXT, to TEXT COLLATE NOCASE, isn't easily done with SQL - that's where SQLtNoCase comes in. http://www.sqlight.com/sqltnocase/ sd _ at _ sqlight _ dot _ com
Re: [sqlite] C precompiler to bytecode
On 8/8/06, Daniel Önnerby <[EMAIL PROTECTED]> wrote: I'm just a bit curios if it would be possible to make like a C precompiler or a macro of some kind that compiles/interpret the SQL-statements to bytecode just like the sqlite_prepare does but does this when compiling/precompiling your application instead of at runtime. Since most application written in C/C++ use static SQL-statements (unless you are building your SQL-strings on the fly) and then bind the values, I guess there would be several benefits for your compiled application: * Faster: Since the SQL-statement is already interpreted. * Smaller (not that sqlite needs to be smaller): The executable does not need to contain the part of sqlite that interprets the SQL-statements since this was made at compile time. Now the cons: * Hardcoded to a specific SQLite version. Even if the *undocumented* API doesn't change on the next version, there's a very good chance it will latter. * Probably not smaller on the easy way.. If you have two pre-compiled statements, it would need a smart pre-compiler to know what can be shared and what can't, implying a level of complexity that maybe doesn't gain nothing. * You can already do the same (with the same problems). By looking at the explain output you can hardcode the same yourself, without the need of a pre-compiler that will need to be constantly up-to-date * If your queries are known at start, you can even forget about SQLite. If you know what you need to do (so, don't need the sqlite library linked, but just a subset of it), maybe it would be best to just call functions and optimize them with a specific data structure that fits you better than sqlite. * Forget about using your program concurrently with others. Future sqlite versions may decide to use different low-level implementations that can clash with your hardcoded functions (this is the same as statically linking the sqlite library with your application). Just my .02 cents... Regards, ~Nuno Lucas
Re: [sqlite] Disable custom collate
On 8/10/06, JP <[EMAIL PROTECTED]> wrote: I created a table with a field and index that uses a custom collation function MYCOLLATE. The problem I am having is portability, I cannot perform simple selects on the table on other sqlite based applications since the custom collation function is not available. Does anybody out ther know of a PRAGMA or directive or method to disable the usage of a custom collation once created? or a way to drop it? I believe the simpler way is to use the ".dump" command and recreate your table from the output. Never tested it (already knew the problem so just kept away from collate), but if it seems to work with some corrupted databases it should work here too. Regards, ~Nuno Lucas jp.
Re: [sqlite] Stripping a newline character
On 9/5/06, Rob Richardson <[EMAIL PROTECTED]> wrote:
Greetings!
I am using a serial communications package that includes a readline()
method to talk to a bar-code scanner. Readline() gives me a string
whose last character is a newline ('\n'). I am trying to get rid of the
newline. MyString.strip('\n') isn't working. It has no effect. How
should I do this?
It seem you pushed the wrong button, but every barcode scanner I
worked with (though I'm certain there are much more different types
out there) had a way to configure the output. So you can just program
the scanner to not send the terminating '\n' (with PS/2 scanners wich
"attach" between the keyboard and the PC I used other function to add
a prefix for the code - like 0x02 [STX] - so I could easily
distinguish codes originated from the scanner and the real keyboard).
Anyway, it seems trivial to just ignore that last '\n'...
Regards,
~Nuno Lucas
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] met "ARM7TDMI raised an exception,data abort" when executing sqlite3Parser() in ARM environment
On 9/11/06, Sarah <[EMAIL PROTECTED]> wrote: During the execution of a test program in an embedded environment, after it goes into sqlite3_exec()-->sqlite3_prepare()-->sqlite3RunParser(, zSql, ), the debugger pops "the ARM7TDMI raised an exception,data abort" when the following sentence is executed(bold and underlined): It can be a misaligned pointer. Try to check what is the value of r0. Also, try to give it a real callback and error pointer. Just to check it's not a bug of sqlite failing to check the parameters are NULL (although I would not bet that it's the reason). Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] met "ARM7TDMI raised an exception,data abort" when executing sqlite3Parser() in ARM environment
[forgot to reply to the list] -- Forwarded message -- From: Nuno Lucas <[EMAIL PROTECTED]> Date: Sep 11, 2006 9:07 PM Subject: Re: Re: [sqlite] met "ARM7TDMI raised an exception,data abort" when executing sqlite3Parser() in ARM environment To: [EMAIL PROTECTED] On 9/11/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: I don't think NULL callback and error pointer will be the reason. Because I've seen this kind of usage in the list for several times. As I said earlier, I don't think this is the reason also. Just one more thing you could check. Anyway, I'll try later and report the result to the list. If it is a misaligned-pointer problem, what can I do? I don't make any change to the sqlite source code, how should this happen? I can run it correctly on PC(windows XP), why can't in an embedded environment? I'm confused... You are aware they are completely different architectures, don't you? You also didn't specify what OS you are running, which could make it easier to get help from other experienced programmers for your platform (if you are using CodeWarrior I would guess it's not Windows CE). A misaligned-pointer can never occur on Windows (desktop editions, not Windows CE), because Windows only runs on the Intel x86 processor family, which mask that kind of things from you (although you usually have a performance hit). Basically it means you are accessing memory that is not aligned to the minimum granularity the processor supports (which depends on the processor and processor mode, but usually is 32 bits - 4 bytes - for 32-bits cpus, 64 bits - 8 bytes - for 64-bits cpus, etc). I don't know if that is your case, but I have seen it before on Windows CE (using a StrongArm processor) and because there aren't so many people using SQLite with those processors, the code path is not so much tested as the x86 case (which doesn't trigger a cpu exception, only a performance hit). It's up to you to confirm this is the case, but there are other things which can be wrong, like little/big endian problems, compiler bugs (recent platforms don't have so much testing as older ones), bad compiler/linker options, stack overflows (because in embedded systems you usually have a much lower default stack size), etc. Without more info that's all I can say. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Summit a bug : data encoded from UTF8 is incorrect
On 9/21/06, 卢炎君 <[EMAIL PROTECTED]> wrote:
Hi all
I use a tool name as sqlitebrowser which from
sourceforge to browsed my db , BTW all data be decoded as utf-8 (lanugage is
chinese), the result of retrive is correct display, so I am sure that all data
has no any problem when be inserted into db. right?
for example, record be stored as utf8 in db as follow:
[-28, -72, -83] "中"
[-27, -92, -82] "央"
[-25, -108, -75] "电"
[-24, -89, -122] "视"
[-27, -113, -80] "台"
[-32, -126, -73] "・"
...(omit other characters)
, when call a function like follow:
const char* function(..)
{
...
return (const char*)sqlite3_column_text(mpVM, nField);
...
}
the return value of result like this:
[-28, -72, -83] "中"
[-27, -92, -82] "央"
[-25, -108, -75] "电"
[-24, -89, -122] "视"
[-27, -113, -80] "台"
[-62, -73, -27] "・" NOTE:value be changed
so I think this is bug
As far as I know sqlite has no bugs in the UTF-8 code, but not all
sqlite clients implement this properly.
SQLite does no checking on the data you use as input (because it can
be binary data, not text) and many GUI's don't make the conversion
to/from UTF-8 before inserting/retrieving data.
Another problem is that the only official sqlite client (the command
line client) doesn't do proper UTF-8 conversion on windows, so windows
users are always faced with this problem one time or another. One
method to test it is if you feed UTF-8 data via an UTF-8 encoded file
instead of using the keyboard. It should work with the file (because
the input is as it should be).
If you only use the C interface to access the database then I only
found UTF-8 support to be wrong when there was a bug in my own code
(as is the case with many sqlite "browsers").
Regards,
~Nuno Lucas
Re: [sqlite] Re: Regarding sqlite3_exec
On 10/25/06, Da Martian <[EMAIL PROTECTED]> wrote:
[...]
But to return all the rows just to count them requires N calls to step. If
the data set is large you only want to return a subset to start with. So you
wouldnt know the count. If you dont know the count, you cant update GUI type
things etc..
There is another alternative if you don't mind to have the overhead of
having an automatic row count (which sqlite avoids by design). It's by
having a trigger that will update the table row count on each
insert/delete.
I'm not good with triggers, but this was already mentioned on the list
and a quick google showed this sample SQL code here:
http://www.northcode.com/forums/archive/index.php?t-6852.html
CREATE TABLE rowcount ( name TEXT, rows INTEGER);
insert into rowcount VALUES ('myTable',0);
UPDATE rowcount SET rows = (SELECT count(myNum) from myTable) WHERE
name = 'myTable';
CREATE TRIGGER incrows AFTER INSERT ON myTable
BEGIN
UPDATE rowcount SET rows = rows+1 WHERE name = 'myTable';
END;
CREATE TRIGGER decrows AFTER DELETE ON myTable
BEGIN
UPDATE rowcount SET rows = rows-1 WHERE name = 'myTable';
END;
After this you can easily access your row count with a
SELECT rows FROM rowcount WHERE name = 'myTable';
Hope this helps,
~Nuno Lucas
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Re: Regarding sqlite3_exec
On 10/25/06, Dennis Cote <[EMAIL PROTECTED]> wrote: Nuno Lucas wrote: > > There is another alternative if you don't mind to have the overhead of > having an automatic row count (which sqlite avoids by design). It's by > having a trigger that will update the table row count on each > insert/delete. > Nuno, This technique only works if you never use conditions on your queries. If your query returns a subset of the rows in a table this carefully maintained count of all the rows in the table is useless. Sure, but I wasn't trying to solve the general issue. The only solution for that is to run the query to full length, whatever the SQL engine is (even if they hide it from the user/programmer). The trigger(s) could be elaborated to specific queries, off course, and that would solve the GUI issue for fixed queries, but the generic case will never have a solution other than the full scan. Anyway, I never used this solution, just trying to show it can be solved for the specific cases of most GUI's, if the trigger overhead is not a problem. In my modest opinion, if one really needs to have an accurate scrollbar, one should show the rows by pages, with buttons to go to the next/prev page (and the scrollbar will be correct per page). No need to know the total rows in the view (unless we know the table doesn't grow that much that a "select count(*)", or a full select into memory, doesn't add much to the window rendering, which is most times). Regards, ~Nuno Lucas Dennis Cote. - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Memory Usage
On 10/27/06, Ben Clewett <[EMAIL PROTECTED]> wrote: I am linking to libsqlite3.so.0.8.6. After calling sqlite3_open(...) I find my programs data memory jumps by 16392 Kb. This seems a lot. The database I am opening is only 26K in size. There are many different ways of memory "jump" (like linking with a lot of dynamic libraries), but one thing is certain: sqlite is not responsible for that. I have a similar process opening about 90 times. This obviously consumes a very large amount of memory, 1.4G with 90 processes. It's a memory leak in your program, for sure. Run some memory leak tool (e.g. valgrind). May I ask if this is what would be expected, and whether there is anything I can do to lower this loading? Unless you decided to mess with sqlite internals, it's not expected in any way. Thanks for your help, Ben. Best regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Memory Usage
On 10/30/06, Ben Clewett <[EMAIL PROTECTED]> wrote: Hi Numo and others, I am very glad to hear the consensus is that there is nothing wrong with libsqlite3.so.0.8.6. However the fact is that the 'open' still acquires 16MB of memory. Immediately Before: VmSize: 8572 kB VmLck: 0 kB VmRSS:2252 kB VmData:484 kB VmStk: 88 kB VmExe: 20 kB VmLib:6772 kB VmPTE: 20 kB Immediately After: sqlite3_open(sDatabaseFile, ) (= SQLITE_OK) VmSize: 24960 kB VmLck: 0 kB VmRSS:2368 kB VmData: 16872 kB VmStk: 88 kB VmExe: 20 kB VmLib:6772 kB VmPTE: 24 kB I guess that info is from /proc//status (the nomeclature somewhat differs for other programs). The program actually only allocated 2368-2252=116 KB, but reserved 16MB of virtual addresses (which is different from actual allocated memory, as you can check by running free or other tool). That is normal if it's the first file you open, as the kernel and libc reserve a bunch of addresses before for internal buffers (to speed up your I/O). RSS (the Resident Set Size), is the important one here (unless your program had parts of it swaped out, which would make it less usefull for what we want). Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Memory Usage
On 10/30/06, Ben Clewett <[EMAIL PROTECTED]> wrote: Nuno, Thanks for the excelent description of my error. I have learnt a little more about Linux virtual memory model. Very glad to hear Sqlite is as perfect as ever :) My problem, which is definitely my problem, is that 90 x 16MB of reserved memory is still a loss of 1.4G. Especially as I use hardly any of it. Each process has it's own virtual address space, so 16MB of reserved virtual addresses (except when they are kernel addresses) for one process doesn't do nothing to the ammount of virtual addresses free for other processes. And as each process usually has 2/3GB of virtual addresses for it's own use, 16MB is pretty low (it depends on the system, but 2 GB is the most common minimum, on 32 bits). I still find strange that your program uses so much virtual addresses, but you didn't specify (or I don't recall) what language you are using and what libraries you are linking to. You may want to investigate this further. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Unicode Help
On 12/7/06, Da Martian <[EMAIL PROTECTED]> wrote:
Ok, that does answer one of my questions I think. If I passed something not
in UTF-8 to sqlite would it return it exactly the same way I passed it in?
From your statement of chaos below I assume it wont if that data somehow
violates UTF-8. So I need to get it to UTF-8 or UTF16 before I insert.
SQLite doesn't care much about what you feed it (remember you can also
have BLOB's in fields), so if you feed it invalid UTF-8, it's invalid
UTF-8 you get on return.
The problem is when you then do do things like "SELECT length("")", or many other text operations. Then you get wrong
results.
The biggest problem is when the database generated by your program is
then read by UTF-8 aware programs (which should be all, but
unfortunely they are not). An example could be an SQLite
importer/exporter program, or some SQLite replicator program you get
on the net and generates bad data because your data wasn't good in the
first place.
Also, i you want to hand edit your data with any of the many good
SQLite GUI's, you may have problems.
If you want to go the simple way (and only do Windows), then use the
UTF-16 functions and forget about all this. As an advantage, windows
NT internals uses Unicode, so you may have some performance gains in
some places (even if negligible most of the time).
Regards,
~Nuno Lucas
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Need a wince test
On 12/20/06, Brodie Thiesfield <[EMAIL PROTECTED]> wrote:
+static HANDLE loadLibraryUtf8(const char *z){
+ WCHAR zWide[MAX_PATH];
+ DWORD dwLen = MultiByteToWideChar(CP_UTF8,0,z,-1,zWide,MAX_PATH);
+ if (dwLen == 0 || dwLen > MAX_PATH) return NULL;
+ return LoadLibraryW(zWide);
+}
I can't test the patch right now, but it's better to use the sqlite
internal UTF-8 to "WideChar" function to MultiByteToWideChar, because
older WinCE versions didn't support CP_UTF8 as parameter (although I
think it was up to the WinCE platform builders to have more "character
encodings" built-in).
Regards,
~Nuno Lucas
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
Re: [sqlite] Need a wince test
On 12/21/06, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote: Uwe Sander <[EMAIL PROTECTED]> wrote: > Hi, > > Am Donnerstag, 21. Dezember 2006 04:03 schrieb [EMAIL PROTECTED]: > > My compiler does not define CP_OEM. Can somebody please tell me > > what the literal value of that macro is? > > I think, the name of the constant is CP_OEMCP, Yes. I figured that out about the same time you sent your email. CP_OEMCP is defined by my compiler. That fixed the problem. Thanks. I don't think Windows CE has any notion of CP_OEMCP. The OEM codepage was the old MS-DOS codepage (the one defined on config.sys) and exists on Windows only for compatibility with old code (and command line applications). A quick browsing over the MSDN site didn't made me sure that it really doesn't exist, but I didn't find any documentation for AreFileApisANSI() on Windows CE. Unfortunately can't test it because I don't have Windows anymore (and the gnu-wince toolchain is still too recent to be an authoritative source). Best regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Question about SQLite for Symbian
On 12/21/06, Joe Wilson <[EMAIL PROTECTED]> wrote: I'd be surprised if a port to a non-mainstream OS was available for free, but you never know. This may be your best bet: If my memory is not playing tricks, I seem to recall some years ago you could download a free development suite, including the Borland C++ Builder 5.0 IDE. Regards, ~Nuno Lucas see 6.0 Custom Modifications http://www.hwaci.com/sw/sqlite/prosupport.html --- panslaw <[EMAIL PROTECTED]> wrote: > I know that SQLite was written in C and I'm sure it's possible to port > it for Symbian OS. But I wonder - maybe there is already (ready to use > )any SQLite library for Symbian OS? It's really importent for to get information > about SQLite and Symbian - thanks in advance for any answers. __ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Mathematical "power" operator?
Just wanted to add that if one of the sqlite GUI's out there uses an external sqlite dll (instead of static linking) it's quite easy to make your own dll with your own functions and replace the one that comes with the GUI application. This was discussed before, but don't know if any one of them actually does this (but an open source one will be easy to recompile that way). Regards, ~Nuno Lucas On 12/21/06, Jeff Godfrey <[EMAIL PROTECTED]> wrote: All, Thanks for the interesting responses. I think I now have a clear understanding of my options, and while not exactly what I was looking for, I can work within the prescribed limitations. As I mentioned, I am working from Tcl, where it's quite easy to write and register a new function with SQLite - which I've done. So, I now have a view containing references to my new "pow" (mathematical power) function, which works exacty as expected from within my own application. Unfortunately, that view now (obviously) causes problems when using 3rd party GUI db manager tools. From here, I think it's just a matter of reorienting my take on the whole problem. As someone already mentioned, SQLite is not intended to be a stand-alone database application. Once I get that fact drilled into my head, I'll be fine... ;^) I will mention that the author of "SQLite Expert" is looking into possible solutions to this specific issue (related to using the new "loadable extensions" feature of SQLite). If anything interesting comes from those discussions, I'll let the list know. Thanks again. Jeff - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Mathematical "power" operator?
On 12/22/06, Jeff Godfrey <[EMAIL PROTECTED]> wrote: Thanks for the nudge. I never thought to check for the external DLL, but checking my SQLite Expert installation folder I see... sqlite3.dll Interestingly, the DLL has the same size and date/time stamp as the one that's available for download on the sqlite.org download page. Well, then you just need to modify src/func.c [1] so that it also "loads" your power function on startup. Maybe looking into how round() is implemented will help you start. Regards, ~Nuno Lucas [1] http://www.sqlite.org/cvstrac/fileview?f=sqlite/src/func.c=1.134 Hmmm ;^) Jeff - To unsubscribe, send email to [EMAIL PROTECTED] - - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] multiuser DB on network share
> Oplocks do not break things. Oplocks will guarantee consistency. They > are granted when only one client OS has a file open letting that client > OS perform locking and caching operations internally without consulting > the server each time. If another client wants to open the file, then > that second open request is held up by the server, the first client > contacted and asked to flush any outstanding data, acquire locks, and > drop the oplock. Once that has happened then the second client gets the > answer to its open request. How is the first client 'contacted' and asked to respond? I can't see how this is anything but useless. I can't imagine very many programs honor this kind of request since I've never even heard of this before last week. If the first client doesn't respond to the request it would have to degenerate to a standard lock. Is this an OS hack designed in for a specific microsoft application? The client is the SMB/CIFS file system driver, not the application. It is all transparent to the programmer, and that is the problem, because if the operating system doesn't handle this well (in other words, is bugged) the programmer has no idea it's corrupting it's own file. One advantage of using a samba server for this is that you can configure each share and even fake oplocks (meaning just ignoring) on read-only media (like a shared CD/DVD drive), meaning substantial performance improvements (the data can be all cached on the user OS). > There are other forms of oplocks that allow for opening and closing the > file multiple times without consulting the server as well as some forms > of limiting sharing (dropped when clients start using byte range locking). > > There have been some problems with Windows when smb signing is in use as > the design of smb signing assumes request response pairs whereas oplock > break notifications are asynchronous. Other than degenerate cases, > current Windows versions have been patched. Degenerate cases? This sounds like something only Microsoft could dream up, so I guess degenerate applies... ;) The whole idea is actually quite clever, but the problem is that it was idealised before people understood everything about networked file systems (the security aspect was completely overlooked at the beginning). The current versions are quite good, but as they have to be compatible with older clients (Win9X), a lot of hacks need to be done (not forgetting it was done in a time Microsoft didn't believe in the future of TCP/IP). For better or worse, is still the major network file system for small networks (and I don't see any future change on this). Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] Obtaining randomness on win32
On 1/29/07, Michael Ruck <[EMAIL PROTECTED]> wrote: On Windows itself you could use the CryptAcquireContext, CryptGenRandom and CryptReleaseContext. I think that doesn't work on WinCE though. It will work on Windows CE since version 2.10, as long as the OS was built with support for it (for those not aware, last Windows CE version is 6.0). A link to MSDN is: http://msdn2.microsoft.com/en-us/library/ms884454.aspx As the link says, if that function is not available, CeGenRandom() can be used instead, but it's only available since Windows CE 4.1. If you intend to use /dev/urandom instead of /dev/random, then I would think CeGenRandom may be enough for the Windows CE implementation (the current sqlite port only compiles out of the box for WinCE 4.x+, anyway). Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] SQLite - how to protect the data
On 2/25/07, mxs <[EMAIL PROTECTED]> wrote: I realize that, but this is about making it difficult for the average Joe not to stop professional hackers who I assume can steal anything they want. Look for SQLITE_FILE_HEADER in the code (it's in the btree.c file). By redefining it, only your application will be able to open a sqlite database created by it. The database will not be encrypted, and you will not be able to open sqlite databases made by other applications, but will be enough for the average Joe. You will have to check every time there is a file format change to not destroy your databases, though. Regards, ~Nuno Lucas - To unsubscribe, send email to [EMAIL PROTECTED] -
Re: [sqlite] UTF16 Encoding
On 2/28/07, Pavan <[EMAIL PROTECTED]> wrote:
Hi,
When i tried to create the db file using API sqlite3_open16("test.db",)
it creats the file name with some junk
characters.
Shouldn't it be "sqlite3_open16(L"test.db",)" ?
Regards,
~Nuno Lucas
The file name is correct if i use sqlite3_open("test.db",). Does it mean
sqlite3 does not support
UTF16 encoding ?
or
Am i missing some package ?
Thanks,
Pavan.
-
To unsubscribe, send email to [EMAIL PROTECTED]
-
