Re: [sqlite] Indexing - a test example

2011-03-07 Thread Black, Michael (IS)
You should learn how to use "explain query plan". Indexes are a trade-off between insert speed (slow) and select speed (fast). If you have a static database there is no such thing as too many indexes as they never would get changed. Most of us live in the world between. Generally, don't

Re: [sqlite] sqlite DB extraction

2011-03-08 Thread Black, Michael (IS)
You go from 2 minutes to 30 minutes just based on clock speed. I would imagine you don't have much disk cache on the ARM device, so every time you read the database it has to hit the file systemouch Did you benchmark your PC system without using a cached database? Just edit the

Re: [sqlite] Optimising a query with several criteria

2011-03-13 Thread Black, Michael (IS)
You don't say how big your database is. My guess is when you see the server using a lot of RAM (and exactly how are you measuring this?) that it's flushing its disk cache. If you're on Unix use vmstat to see what your OS cache is doing. So...perhaps if you increase SQLite's internal cache it

Re: [sqlite] Optimising a query with several criteria

2011-03-13 Thread Black, Michael (IS)
: Sunday, March 13, 2011 8:40 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] Optimising a query with several criteria Hi Michael, thanks for this. My database is 40 megabytes (and growing slowly) - is that a reasonable cachesize? On 13/03/2011 13:07, Black, Michael

Re: [sqlite] Optimization SQLite

2011-03-14 Thread Black, Michael (IS)
Try CodeBlocks Cross-platform and works with gcc or MSVC or pretty much whatever. I put in gcc 64-bit for it. http://www.codeblocks.org/ Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From:

Re: [sqlite] Newbie question

2011-03-21 Thread Black, Michael (IS)
Try this. BEGIN TRANSACTION; CREATE TABLE sales(SaleDate date,SaleVolume int); INSERT INTO "sales" VALUES('2010-01-01', 10); INSERT INTO "sales" VALUES('2010-01-02', 20); INSERT INTO "sales" VALUES('2011-01-01', 15); INSERT INTO "sales" VALUES('2011-01-02', 30); INSERT INTO "sales"

Re: [sqlite] Bulk loading large dataset; looking for speedups.

2011-03-22 Thread Black, Michael (IS)
But I thought he said he dropped the indexes (meaning they aren't there during inserts). That should make sorting irrelevant. 3 Things. #1 Test with :memory: database and see what the speed is. That tells you if it's SQLite or disk I/O as the bottleneck. #2 Try WAL mode "pragma

Re: [sqlite] storing big numbers into NUMERIC, DECIMAL columns

2011-03-22 Thread Black, Michael (IS)
I hope you know what you're doing with trying to preserve that much significance. Ths first time you stick it in a double or long double variable you'll lose it. You can use the HPAlib to get 32 digits http://www.nongnu.org/hpalib/ // Example showing digit loss -- doesn't matter double or

Re: [sqlite] storing big numbers into NUMERIC, DECIMAL columns

2011-03-22 Thread Black, Michael (IS)
Hmmm...the docs do say that...but how do you get that value back out? Retreiving it as text doesn't work. You still don't say what you're planning on doing with these number...just displaying them? I think the docs may be misleading...here is the comment in sqlite3.c /* ** Try to convert a

Re: [sqlite] storing big numbers into NUMERIC, DECIMAL columns

2011-03-23 Thread Black, Michael (IS)
Blob may be better if you need speed -- then no conversion is necessary inside your Pascal code to/from a string. But if you want to be able to see and understand your database text is better (or you have to write a special Pascal program to decode your database to look at any problems).

Re: [sqlite] Reg: Link error while using sqlite 3

2011-03-23 Thread Black, Michael (IS)
Unless you're running multiple SQLite apps you don't gain anything by using a DLL. Plus, is your Pocket PC a i386 CPU? So try downloading the amalgamation and include sqlite3.c and sqlite3.h in your project. http://www.sqlite.org/sqlite-amalgamation-3070500.zip You'll also find the code will

Re: [sqlite] storing big numbers into NUMERIC, DECIMAL columns

2011-03-23 Thread Black, Michael (IS)
Hmmm...according to my math... Max 64-bit unsigned integer is 18446744073709551615 Drop the last digit as it can't hold 0-9 1844674407370955161 Make two decimal positions 18446744073709551.61 Now some commas so we can see better 18,446,744,073,709,551.61 That' $18 quadrillion dollars by my

Re: [sqlite] Full Table Scan after Analyze

2011-03-26 Thread Black, Michael (IS)
When you say "All an index does" don't forget that an index is also usually smaller than the data, thereby increase cache performance and reducing disk seeks. For a good chunk of typical uses (large tables with simple lookups) an index is notably faster. I'll admit my use of sqtlite3 hasn't

Re: [sqlite] Lack of "decimal" support

2011-03-27 Thread Black, Michael (IS)
Base 10 multiplication is needed. Although money is the main reason for doing BCD due to cumulative errors it's not the only reason. So I'd recommend adding decimal*decimal -> decimal just to be complete. It's easy enough to implement using the + function so could just be noted as "slow".

Re: [sqlite] sqlite3_step behavior on empty set

2011-03-28 Thread Black, Michael (IS)
Wouldn't the addition of "Empty result sets will return SQLITE_DONE on the first call to sqlite3_step." add some clarity? Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org

Re: [sqlite] bug with sqlite

2011-03-29 Thread Black, Michael (IS)
Just to help clarify (hopefully) the Unix/Windows "reserved filename". CON: is similar to Unix's /dev/zero or /dev/null for example -- Files that already exist and have OS meaning. stdout is NOT a reserved filename...it's a predefined variable of FILE *. You cannot say "cp file stdout" on

Re: [sqlite] Generate a unique id unless it is provided

2011-03-29 Thread Black, Michael (IS)
I don't understand how your column C works...so I'll assume it's pre-known for now. But here's how to get A/B to work. drop table t if exists; create table t(a int,b int,c int); create trigger trig1 after insert on t begin update t set a=(select count(b) from t where b=new.b) where a=0; end;

Re: [sqlite] mutex assert_fail in btreeInvokeBusyHandler occasionally in a periodic DB update in 3.5.7, It's ok in 3.6.22(-DSQLITE_THREADSAFE=1)

2011-03-31 Thread Black, Michael (IS)
Since it apperas you're running your commit in a separate thread and are therefore muilti-threaded I do belive you need: SQLITE3_THREADSAFE=2 >From http://www.sqlite.org/compile.html#threadsafe To put it another way, SQLITE_THREADSAFE=1 sets the default threading mode to Serialized.

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char

2011-04-03 Thread Black, Michael (IS)
You apparently don't understand "strings" in C. Or are you actually reading in binary data? #1 Since you said "image" I assume you're reading binaary. So get rid of buffer[fsize]=0. You don't null terminate binary data and that statement is 1-beyond the end of the array (which is from 0 to

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char

2011-04-03 Thread Black, Michael (IS)
How are you trying to view the ouitput.result.txt (and I"ll note that it'sNOT a text file...it's an image according to what you said.). What's the size of the file. And you should be able to post a COMPLETE example to show your testing. What you say you want to do has been done by many

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char - FULL CODE

2011-04-03 Thread Black, Michael (IS)
I ran your code with my test file and I get this...which is perfectly correct. Do you get something different? What makes you think the stream is truncated in the database? Also...change SQLITE_STATIC to SQLITE_TRANSIENT...that could be your culprit if you are still seeing truncation. Plus

[sqlite] :Re: sqlite3_bind_blob CHOPS off at first NULL char

2011-04-03 Thread Black, Michael (IS)
l 003 I must be doing something fundamentally wrong.;-( Thanks for the help so far, I really appreciate it..... Lynton On 03/04/2011 13:44, Black, Michael (IS) wrote: > How are you trying to view the ouitput.result.txt (and I"ll note that it'sNOT > a text file...it's an image

Re: [sqlite] sqlite-amalgamation + own functions = Segmentation fault at "sqlite3VdbeMemSetStr" sice version 3.7.5 (to v3.7.3 works fine)

2011-04-16 Thread Black, Michael (IS)
Care to show us your SPL_mallocstr() function? Sounds like you've corrupted data if that line dies. All it's doing is checking an array value which is used all over the place in sqlite3.c If you put a break point there and on the first time it's hit put a watch on the address for

Re: [sqlite] 'integer'

2011-04-17 Thread Black, Michael (IS)
Seems to behave OK for me on 3.7.5 on Windows. What version are you using on what OS with what compile flags? You also "said" it didn't work but you didnt' actually what what you did. Like this... SQLite version 3.7.5 Enter ".help" for instructions Enter SQL statements terminated with a ";"

Re: [sqlite] 'integer'

2011-04-18 Thread Black, Michael (IS)
That's the nice thing about standards...there's so many to choose from :-) Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Simon

Re: [sqlite] What happens if you insert more than your RAM size into an in memory database?

2011-04-18 Thread Black, Michael (IS)
Ummm...are we forgetting about swap space? If you exceed RAM you hit swap. If you exceed RAM+SWAP you start failing. Or does sqlite monitor physical memory usage? So if you exceed RAM you just start slowing down towards a disk-based equialent database. Michael D. Black Senior

Re: [sqlite] Request for help with SQLite Query to return missing Date/Time Ranges

2011-04-21 Thread Black, Michael (IS)
Assuming your database ONLY contains the log entries this should workand be pretty fast too since rowid is already indexed and there areYou no other lookups. You can add your own rowid to make this work otherwise. Just do a max(myrowid)+1 on your insert. PRAGMA foreign_keys=OFF; BEGIN

Re: [sqlite] date field with default current date

2011-04-21 Thread Black, Michael (IS)
create table t (d default CURRENT_DATE,i number); insert into t (i) values(1); select * from t; 2011-04-21|1 Use CURRENT_TIME if you want hours/minutes too. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From:

Re: [sqlite] Request for help with SQLite Query to return missing Date/Time Ranges

2011-04-21 Thread Black, Michael (IS)
A slight mod on my solution makes it work for DST changes too. Again...rowid must be maintained. PRAGMA foreign_keys=OFF; BEGIN TRANSACTION; CREATE TABLE log(d date); INSERT INTO "log" VALUES('2011-03-13 01:55'); INSERT INTO "log" VALUES('2011-03-13 01:56'); INSERT INTO "log"

Re: [sqlite] EXT :Re: BUG : round(x,y) not consistent

2011-04-30 Thread Black, Michael (IS)
3.75 on Redhat 5.6 does not have a problem. I get 8.88 16.88 32.88 64.88 Fixed in 3.7.5 maybe? Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org

[sqlite] RE : Feature request: Fuzzy searching

2011-05-03 Thread Black, Michael (IS)
Sounds like soundex might work for you. Not sure how it handles diacritics though. http://www.mail-archive.com/sqlite-users@sqlite.org/msg35316.html Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From:

Re: [sqlite] EXT : Transaction speed too slow?

2011-05-09 Thread Black, Michael (IS)
Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Nick [maill...@css-uk.net] Sent: Sunday, May 08, 2011 3:52 PM To:

Re: [sqlite] Transaction speed too slow?

2011-05-09 Thread Black, Michael (IS)
You do realize the number they quote is a MAXnot necessarily what you'll get. With 16 transactions per second you're writing on transaction every 63ms. My understanding is that the hard drive in the netbook is 15ms access time so you're seeing 4 accesses per transaction with the way

Re: [sqlite] Memory leak in SQlite

2011-05-10 Thread Black, Michael (IS)
#1 I don't see where you're freeing m_szErrorString (not real sure if it gets malloc'd on success) -- but you do need to free it on errors for sure. And where's your Callback function? Why are you calling SaveResultSet (which you also don't show)? That should probably be done inside the

Re: [sqlite] Blob newbie problem

2011-05-12 Thread Black, Michael (IS)
Do I understand you're still seeing a segfault? I assume you're not seeing your "Year retrieved..." statement? You haven't showed us your table definition. Change your strcmp to strcasecmp and see if that fixes it for you. Michael D. Black Senior Scientist NG Information Systems

Re: [sqlite] Cannot load SQLite.Interop.dll but file is in the folder

2011-05-29 Thread Black, Michael (IS)
Welcome to DLL hell...Microsoft keeps changing it... http://msdn.microsoft.com/en-us/library/ms682586(v=vs.85).aspx Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org

Re: [sqlite] Cannot load SQLite.Interop.dll but file is in the folder

2011-05-31 Thread Black, Michael (IS)
as the EXE file. I do not know if there is a link but the user who experiences this problem is running my application under WIndows 7 32bit. Is there any possible link? Thanks again Cyrille Le 29/05/2011 20:33, Black, Michael (IS) a écrit : > Welcome to DLL hell...Microsoft keeps changing it... >

Re: [sqlite] Do I need to migrate to MySQL?

2011-06-03 Thread Black, Michael (IS)
My radar says having a TEXT field as a primary key is bad (your userTable). String compares are horrendously slow. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org

Re: [sqlite] Do I need to migrate to MySQL?

2011-06-03 Thread Black, Michael (IS)
And do you wrap all your updates inside a transaction? Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Ian Hardingham

Re: [sqlite] Do I need to migrate to MySQL?

2011-06-03 Thread Black, Michael (IS)
return 0; } return 0; } On 03/06/2011 15:28, Black, Michael (IS) wrote: > > And do you wrap all your updates inside a transaction? > > Michael D. Black > > Sen

Re: [sqlite] Do I need to migrate to MySQL?

2011-06-03 Thread Black, Michael (IS)
} else { // error occured Con::executef(this, 2, "onQueryFailed", m_szErrorString); delete pResultSet; return 0; } return 0; } On 03/06/2011 15:28, Black, Michael (IS) wrote: > > And do you wrap all your updates inside a transactio

Re: [sqlite] Handling of IEEE-754 nan and +/-inf in SQLite?

2011-06-06 Thread Black, Michael (IS)
That's the nice thing about standards...there's so many to choose from... :-) Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of

Re: [sqlite] To index or not to index?

2011-06-13 Thread Black, Michael (IS)
Ummm...wouldn't it be a whole lot simpler and faster to have an "Online" table? One id is all that's needed unless you want other info. Then you're just inserting and deleting records and the whole table is "online" at any moment. I don't see any reason why this online status needs to be

Re: [sqlite] A simple SELECT

2011-06-13 Thread Black, Michael (IS)
You do this: INVOKE sqlite3_prepare_v2,__hSql,__lpszQuery,-1,eax,edx Then this: INVOKE sqlite3_step,_lpSQLStatment You probably want INVOKE sqlite3_prepare_v2,__hSql,__lpSQLStatement,-1,eax,edx At least from what I tell of your logic. You're not using the same variable fo the statement

Re: [sqlite] A simple SELECT

2011-06-13 Thread Black, Michael (IS)
Oops...sorry...you're correct...been far too long since I've done assembler. Are we to understand that if you simply give this table a different name it works? That seems to be what you said in your initial email. Are you inserting records in the table in your assembler too? Are you

Re: [sqlite] EXT :Re: A simple SELECT

2011-06-13 Thread Black, Michael (IS)
...@phrio.biz] Sent: Monday, June 13, 2011 12:08 PM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] A simple SELECT If I change the table name it works. I tryed the same query using SqLite3Explorer and it does not give any result ! -Message d'origine- From: Black, Michael

Re: [sqlite] A simple SELECT

2011-06-13 Thread Black, Michael (IS)
) [m...@phrio.biz] Sent: Monday, June 13, 2011 12:45 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] EXT :Re: A simple SELECT For example "Test" -Message d'origine----- From: Black, Michael (IS) Sent: Monday, June 13, 2011 7:30 PM To: General Discussion of SQLit

Re: [sqlite] need help with a query using datetime

2011-06-17 Thread Black, Michael (IS)
sqlite> create table t(d date); sqlite> insert into t values('2011-12-31 09:00'); sqlite> insert into t values('2011-12-31 12:15'); sqlite> select d,substr(datetime(d,'-12 hours'),1,16) from t; 2011-12-31 09:00|2011-12-30 21:00 2011-12-31 12:15|2011-12-31 00:15 Michael D. Black Senior

Re: [sqlite] Critical issue

2011-06-18 Thread Black, Michael (IS)
See if this thread helps you -- a couple of different solutions. Both of which you should be able to solve for your users. http://forums.asp.net/t/939729.aspx/1 Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate

Re: [sqlite] Critical issue

2011-06-18 Thread Black, Michael (IS)
Also...try dependency walker to ensure you have all the DLLs you need deployed with your package that aren't "standard". http://www.dependencywalker.com/ Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From:

Re: [sqlite] Critical issue

2011-06-19 Thread Black, Michael (IS)
questions are stupid but I cannot figure it out. Best regards, Cyrille Le 18/06/2011 15:50, Black, Michael (IS) a écrit : > Also...try dependency walker to ensure you have all the DLLs you need deployed with your package that aren't "standard". > > http://www.dependencywalk

Re: [sqlite] Help sqlite database corruption

2011-06-21 Thread Black, Michael (IS)
As I said before...welcome to DLL hell... You guessed itthe original DLL will give an error if any of the DLLs it depends on are not loadable. Since sqlite.interop.dll is non-standard you should include ALL DLLs that it depends on in your installation. Subject to Microsoft's

Re: [sqlite] Howto...multi-sqlite command string through sqlite3_prepare_v2() to create SINGLE statement

2011-06-21 Thread Black, Michael (IS)
I believe this will work if you put the SQL-required semi-colons at the end of your statements. sql = "BEGIN;"; //you need to add newline here sql += "create table episodes (id integer primary key, season int, name text);"; sql += "insert into episodes(id, season, name) Values(1,2,'bill');";

Re: [sqlite] Query with UNION on large table

2011-06-23 Thread Black, Michael (IS)
Any reason you can't add another field to your database? 0=equal 1=n1n2 The create an index on that field. Then your query plan would look like this: sqlite> explain query plan select n1,n2 from table2 where flag = 1 intersect select n2,n1 from table2 where flag = 2; sele order

Re: [sqlite] Query with UNION on large table

2011-06-23 Thread Black, Michael (IS)
[sqlite-users-boun...@sqlite.org] on behalf of Black, Michael (IS) [michael.bla...@ngc.com] Sent: Thursday, June 23, 2011 2:03 PM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] Query with UNION on large table Any reason you can't add another field to your database? 0=

Re: [sqlite] question about sqlite aggregate funcitons

2011-06-25 Thread Black, Michael (IS)
Ucaching? Try doing your first select last and see what happens. I'm betting your first "select min(*) from test" will be a lot slower. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From:

Re: [sqlite] question about sqlite aggregate funcitons

2011-06-25 Thread Black, Michael (IS)
NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Luuk [luu...@gmail.com] Sent: Saturday, June 25, 2011 7:37 AM To: sqlite-users@sqlite.org Subject: EXT :Re: [sqlite] ques

Re: [sqlite] question about sqlite aggregate funcitons

2011-06-25 Thread Black, Michael (IS)
s-boun...@sqlite.org] on behalf of Black, Michael (IS) [michael.bla...@ngc.com] Sent: Saturday, June 25, 2011 8:15 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] question about sqlite aggregate funcitons OK...I generated 10M ints and imported them. I get times that I would

Re: [sqlite] question about sqlite aggregate funcitons

2011-06-25 Thread Black, Michael (IS)
Can the optimizer recognize these "instant" functions and perhaps lean towards using them? Are there cases where this would NOT be good? I did show you can construct a query which runs at the "correct" speed. sqlite> select * from (select min(i) from test) as a,(select max(i) from test)

Re: [sqlite] EXT :Re: question about sqlite aggregate funcitons

2011-06-25 Thread Black, Michael (IS)
Slavin [slav...@bigfraud.org] Sent: Saturday, June 25, 2011 10:38 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] question about sqlite aggregate funcitons On 25 Jun 2011, at 4:12pm, Black, Michael (IS) wrote: > Can the optimizer recognize these "instant&qu

Re: [sqlite] Substring question

2011-06-26 Thread Black, Michael (IS)
It's not obvious but this works CREATE TABLE x (s string); INSERT INTO "x" VALUES('ab:cdef'); INSERT INTO "x" VALUES('ghij:klmn'); sqlite> select ltrim(ltrim(s,'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'),':') from x; ltri cdef klmn Just make sure your char set contains all

Re: [sqlite] Compile sqlite3 with g++? Do I need a wrapper?

2011-06-28 Thread Black, Michael (IS)
I'd recommend NOT relying on the system sqlite3. That way you can control your changes. Get the amalgamation and put sqlite3.c and sqlite3.h in your project. And, you forgot to put in the name for "-o" -- so you would get a file named "-lsqlite3" in your directory. And you'll probably

Re: [sqlite] Is this a BUG or am I wrong ? sub-select returns nothing when column id matches the inner query

2011-06-28 Thread Black, Michael (IS)
Use single quotes instead of double sqlite> select (select v from t1 where n='a') wrong,* from a1; wrong|a|b 2000|123|456 2000|999|999 Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From:

Re: [sqlite] Defining a relationship as unique

2011-06-30 Thread Black, Michael (IS)
You're getting closeif you don't use a field in a table you don't HAVE to create it. Also...if you want to make your database a bit more bullet proof you want foreign keys to help ensure you maintain the relationship between users and tournaments (otherwise you can accidentally delete

Re: [sqlite] Defining a relationship as unique

2011-06-30 Thread Black, Michael (IS)
e: [sqlite] Defining a relationship as unique On 06/30/11 02:31 PM, Black, Michael (IS) wrote: > sqlite> create table user(userid integer primary key autoincrement,name > varchar, login varchar); > sqlite> create unique index index1 on user(userid); Isn't userid already unique by &q

Re: [sqlite] How to search for multiple values in a column including null without repeating column name ?

2011-07-02 Thread Black, Michael (IS)
If you can build your SQL string with column_name exactly what is the problem with adding it a 2nd time? How do you build your SQL statement? And I get the same answer with either of your queries in 3.7.4 -- what version are you running and what answer are you getting? Are you saying that

Re: [sqlite] How to search for multiple values in a column including null without repeating column name ?

2011-07-02 Thread Black, Michael (IS)
That would 'splain it...I should have noticed... So we're left with this question... What's the problem with building the query that works? I don't see where you can't make it "column agnostic"...though perhaps there's another definition of that sprintf(sql,"select * from t where %s

Re: [sqlite] Insert not working for sqlite3

2011-07-07 Thread Black, Michael (IS)
You're going to have to create a complete stand-alone example if you want us to debug your code for you. Make a simple program that only does that one insert and share that with us. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate

Re: [sqlite] SQLite Encryption Extension (SEE) and Public Domain Sqlite

2011-07-07 Thread Black, Michael (IS)
If you hash the username too you don't have that problem. Passwords should always be a 1-way hash. It's actually best to collapse username/password into a single one-way hash. That way it's very difficult to crack it. You can use a user "account#" value that users would have to track for

Re: [sqlite] Insert not working for sqlite3

2011-07-08 Thread Black, Michael (IS)
I downloaded 5.3.6 from here -- I used the Thread Safe version Installer. http://windows.php.net/download/ And you're script ran just fine and created this: sqlite> .dump PRAGMA foreign_keys=OFF; BEGIN TRANSACTION; CREATE TABLE tbl2 (one varchar(10),two varchar(10)); COMMIT; I the modified

Re: [sqlite] using index when using concatination

2011-07-10 Thread Black, Michael (IS)
Any particular reason you can't build your own string and just pass one parameter? select * from words where "word" like ?; And any reason why you don't create a 2nd field holds the 1st and last char? and index that? Then your query should be blazingly fast as it will actually use the index

Re: [sqlite] using index when using concatination

2011-07-10 Thread Black, Michael (IS)
3.7.4 doesn't indicate it will use an index in either case with like...but glob seems to call the index...why does glob use the index but like does not? sqlite> create table words(word text); sqlite> create index idx on words(word); sqlite> explain query plan select * from words where "word"

Re: [sqlite] using index when using concatination

2011-07-10 Thread Black, Michael (IS)
te> select count(*) from words where word like 'L%P'; 136 CPU Time: user 0.281250 sys 0.00 Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@s

Re: [sqlite] Storing/editing hierarchical data sets

2011-07-10 Thread Black, Michael (IS)
Somebody smarter than I may be able to figure out how to use views to do the upper levels. But if you can afford your database to be a bit less then twice as big just use tables. create table level1(id int,l int,r int); insert into level1 values(1,251,18); insert into level1 values(2,5,91);

Re: [sqlite] Insert not working for sqlite3

2011-07-11 Thread Black, Michael (IS)
If CREATE has no return why do I get a return? To be clear I'm using PHP 5.3.6 and it most definitely gets a return But I re-wrote this according to some PDO examples I found. This is a more complete example. You should be able to run this with a PHP command line -- forget the web

Re: [sqlite] Storing/editing hierarchical data sets

2011-07-11 Thread Black, Michael (IS)
I can see adding a forward/reverse link to the tables making it a linked-list type structure much like your btree. By default each node is linked to the one in front and back. Then you adjust those pointers for cut/paste operations. You could also do the cut/paste just by copying to a new

Re: [sqlite] Storing/editing hierarchical data sets

2011-07-12 Thread Black, Michael (IS)
I thought of another way to do your copy/cut/paste... Assuming you keep the original audio around and use the levels I showed before. create table sequence(level int,parent int,start int,end end); insert into seqeunce values(1,0,0,-1); // note that -1 means "until end of data". See

Re: [sqlite] Minimize sqlite3.o size

2011-07-12 Thread Black, Michael (IS)
http://www.sqlite.org/compile.html Read section 1.4 -- tells you all you need to know. First thing you should do is see if there's any significant performance difference with -Os instead of -O3. You may be surprised. Michael D. Black Senior Scientist NG Information Systems Advanced

Re: [sqlite] Index question about index

2011-07-14 Thread Black, Michael (IS)
Would glob be faster than like? Since I assume like has to case-convert? And you don't have to set any pragmas or NOCASE for it to use the index. select * from tsamov where tsamov_code glob 'AFG*'; sqlite> explain query plan select * from tsamov where tsamov_code glob 'AFG*'; sele order

Re: [sqlite] Hidding records from the application

2011-07-16 Thread Black, Michael (IS)
If each pid sees a completely separate set of rows then add the pid to each row and select by pid. If 2 pid's can see any of the same records this won't work (i.e. all pid's must be mutually exclusive) No views necessary but you could create a view for each pid if you wanted to for some

Re: [sqlite] year, month & day problem

2011-07-17 Thread Black, Michael (IS)
Column numbers are zero-based, not one-based. >From http://www.sqlite.org/c3ref/column_blob.html "The leftmost column of the result set has the index 0. " That's at least part of your problem. So should be: k.AAArticoli=sqlite3_column_int(statmentS, 0);

Re: [sqlite] Hidding records from the application

2011-07-17 Thread Black, Michael (IS)
If you don't know how to get your rules yet then you don't know how to design a solution. Based on what you've said I see 2 more tables. create table (pid int, rule int) create table (rule int, record int) You can have multiple rules per pid, reuse rules across pids, and records can

Re: [sqlite] Hidding records from the application

2011-07-17 Thread Black, Michael (IS)
I decided to test this so here's an almost-complete example minus any cross-checks on the tables which you should ultimately do. This implements exclusion rules which is what you've been describing. You can make them inclusion rules if you just remove the "not" in the select statement below, but

Re: [sqlite] Handle leak using IIS on windows 7?

2011-07-18 Thread Black, Michael (IS)
If you showed a small sample code that caused your problem it would sure help a lot. I peeked at the library code. That message is generated when _cnn is null (the DB connection). private void InitializeForReader() { if (_activeReader != null && _activeReader.IsAlive)

Re: [sqlite] Performance Improvement

2011-07-18 Thread Black, Michael (IS)
Unless your caching at byte-level you don't need all the addresses. For 4K page size mask off the lower 11 bits. You should actually be able to reduce your memory usage by a LOT if you track by pages and not bytes. That will also speed you up along with it. Michael D. Black Senior

Re: [sqlite] Performance Improvement

2011-07-21 Thread Black, Michael (IS)
Could you define "awfully slow"? That's pretty hard to tell if your speed is what one should expect. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Paul Linehan

Re: [sqlite] Select names where occurences are greater than 10

2011-07-21 Thread Black, Michael (IS)
select id,count(distinct track) from tracks group by id having count(distinct track)>=10; Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on

Re: [sqlite] SELECT query first run is VERY slow

2011-07-22 Thread Black, Michael (IS)
You need to normalize your database. Though it's easy to put everything in one table it's horrendously inefficient for indexing. Your indexes are huge as you're putting strings in them. Ideally you should never have a string in an index if you can avoid it. Too much space, too long a

Re: [sqlite] Strange result using JOIN

2011-07-22 Thread Black, Michael (IS)
Try "order by code.rowid" I think that will do what you want and fits in with what sqlite3's behavior should be. As long as you don't delete rows you'll be OK. http://www.sqlite.org/autoinc.html Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From:

Re: [sqlite] System.Data.Sqlite problem

2011-07-22 Thread Black, Michael (IS)
And don't you wish Microsoft and other's could spit out intelligent error messages like: "64-bit application trying to load 32-bit DLL". I ding my students whey do stupid stuff like: FILE *fp=fopen(filename,"r"); if (fp == NULL) { fprintf(stderr,"Cannot open file\n"); } Instead

Re: [sqlite] EXT :Re: SELECT query first run is VERY slow

2011-07-25 Thread Black, Michael (IS)
An 8X speedup is a pretty good achievement...congrats... #1 I take it your query is CPU bound the first time? #2 Can you show us the query planner please? #3 Can you show us the query planner minus the "INDEXED BY"? #4 Can you show us sqlite_stat1? #5 Can you show us your tables now? #6

Re: [sqlite] SELECT query first run is VERY slow

2011-07-25 Thread Black, Michael (IS)
You need to normalize your "kind" value. .pragma cache_size=15000; drop index idxlog_kind_computer; create table kind(id integer,kind text); insert into kind values(1,'debug'); insert into kind values(2,'error'); insert into kind values(3,'info'); insert into kind values(4,'timing'); insert

Re: [sqlite] SELECT query first run is VERY slow

2011-07-26 Thread Black, Michael (IS)
That's because they have 2 completely different query plans. I created the table so that id,a,b,c all had the same values so the indexing would be indentical. #include main() { int i; for(i=1;i<=10;++i) { char sql[4096]; sprintf(sql,"insert into abctable(a,b,c)

Re: [sqlite] SELECT query first run is VERY slow

2011-07-26 Thread Black, Michael (IS)
If I run your sql.txt script with the normalization of kind my first time query shows 0 seconds. D:\x>sqlite3 sq1 > select count(*) as cnt,kind,computer > from log > group by kind,computer > order by cnt desc > > what happens here? > > SELECT * > FROM log INDEXED BY idxlog_kind_computer >

Re: [sqlite] SELECT query first run is VERY slow

2011-07-26 Thread Black, Michael (IS)
Part of the problem is it seems you can't create an index with rowid: 3.7.5 sqlite> create table t(i int); sqlite> create index idx1 on t(i); sqlite> create index idx2 on t(i,rowid); Error: table t has no column named rowid Any particular reason it can't be included in an index?

Re: [sqlite] LevelDB benchmark

2011-07-29 Thread Black, Michael (IS)
What they don't say explicitly is that if all you need is key/value capability then an SQL database is overkill and only slows you down (bit of a duh factor there though not obvious to neophytes). Generally speaking that's one thing they don't teach in college is optimization. I can't count

Re: [sqlite] LevelDB benchmark

2011-07-29 Thread Black, Michael (IS)
bject: EXT :Re: [sqlite] LevelDB benchmark 2011/7/29 Black, Michael (IS) <michael.bla...@ngc.com>: > What they don't say explicitly is that if all you need is key/value > capability then an SQL database is overkill and only slows you down (bit of a > duh factor there though

Re: [sqlite] LevelDB benchmark

2011-07-30 Thread Black, Michael (IS)
I ran a few tests. First the 2 tests from Rev 45. Then NORMAL sync, and finally bump the cache up to what sqlite3 needs vs an arbitrary 100M. I think the question should be how well does it perform in cache -- not necessarily how much cache it needs. How much cache is just a comparision of

Re: [sqlite] Will SQLite supports UnQL?

2011-08-01 Thread Black, Michael (IS)
This is a side-question to this thread...but has anybody every done row-level locking for edit? I can see it: create table t(id int primary key,stuff text, lock l); insert into t values(1,'stuff1',0); select * from t where id=1 and lock=0; // or drop lock to get all and check lock!=0 to

Re: [sqlite] Virtual Table xBestIndex and NULL Search Conditions (Bug?)

2011-08-01 Thread Black, Michael (IS)
If it's meaningless then shouldn't it be a syntax error? Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Simon Slavin

  1   2   3   4   5   6   7   8   9   >