Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-12 Thread Black, Michael (IS)
qlite-users-boun...@sqlite.org] on behalf of Jim Wilcoxson [pri...@gmail.com] Sent: Saturday, February 12, 2011 10:11 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected On Sat, Feb 12, 2011 at 9:10 AM, Black, Michael (IS) wrote: &

Re: [sqlite] EXT :Re: UPDATE/INSERTing 1-2k rows slower than expected

2011-02-12 Thread Black, Michael (IS)
9:10 AM, Black, Michael (IS) > > D:\SQLite>batch 5000 1 >> 360766.6 inserts per sec >> > > Unless I'm missing something, SQLite has to update the first page of the > database on every commit, to update the change counter. Assuming you are > using rotating media,

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-12 Thread Black, Michael (IS)
OK...I added your trigger example as option 8. And I had pasted the wrong version in my last email. My timings were correct. Your example also did sql_exec instead of using prepare so it will run slower. I also made this compilable on Unix too. On Unix my timing matches the run time and there

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-14 Thread Black, Michael (IS)
SOLVED!! Marcus Grimm and I went back forth a number of times trying to figure out why my benchmarks were so much faster than his. Found it... My SATA RAID setup had "Enable Advanced Performance" on by default (I had never turned it on). My secondary tests on an IDE drive showed similar perfor

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-14 Thread Black, Michael (IS)
Also...here's another part of the benchmark which shows triggers aren't as evil as I thought. Trigger for this example was 2X the manual update. F:\>d:batch 50 1000 12 using wal mode using update trigger Sqlite Version: 3.7.5 Inserting 50 rows using a bulk of 1000 commits per second: 714

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-14 Thread Black, Michael (IS)
And you think Jim's timings are wrong because.. I've already shown you can get speed just like he's showing. That's what you get on a good write-cache-enabled drive. And if you want to talk about data reliability...BACK UP YOUR DATA. The likely failure points I can think of are: #1 Power

Re: [sqlite] EXT :Re: UPDATE/INSERTing 1-2k rows slower than expected

2011-02-14 Thread Black, Michael (IS)
PM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected On Mon, Feb 14, 2011 at 11:50 PM, Black, Michael (IS) < michael.bla...@ngc.com> wrote: > And if you want to talk about data reliability...BACK UP YOUR DATA. > The likely fai

Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected

2011-02-15 Thread Black, Michael (IS)
v...@bigfraud.org] Sent: Monday, February 14, 2011 5:04 PM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] UPDATE/INSERTing 1-2k rows slower than expected On 14 Feb 2011, at 8:50pm, Black, Michael (IS) wrote: > And if you want to talk about data reliability...BACK UP YOUR DATA.

Re: [sqlite] Performance Problem

2011-02-16 Thread Black, Michael (IS)
Try this benchmark program and see what numbers you get. You need to compare to other machines with the same benchmark to see if it's the machine or your programming/architecture. The MC55 is a 520Mhz PXA270 so I would expect to see more than a 6X difference from my 3Ghz box (memory speed is no

Re: [sqlite] Compiler warnings in R-Tree code under Visual StudioExpress

2011-02-18 Thread Black, Michael (IS)
I'm of the opinion that all such warnings should be permanently fixed. Such warnings do point to potential problems. And not by disabling the warning but by fixing the code (explicit casts for example). How many people try this and get worried about possible problems? If you simply fix the co

Re: [sqlite] EXT :Re: Compiler warnings in R-Tree code under Visual StudioExpress

2011-02-18 Thread Black, Michael (IS)
, February 18, 2011 7:53 AM To: General Discussion of SQLite Database Cc: Black, Michael (IS) Subject: EXT :Re: [sqlite] Compiler warnings in R-Tree code under Visual StudioExpress On Fri, Feb 18, 2011 at 8:12 AM, Black, Michael (IS) mailto:michael.bla...@ngc.com>> wrote: I'm of the

Re: [sqlite] Compiler warnings in R-Tree code under Visual StudioExpress

2011-02-19 Thread Black, Michael (IS)
users-boun...@sqlite.org] on behalf of Roger Binns [rog...@rogerbinns.com] Sent: Friday, February 18, 2011 7:45 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] EXT :Re: Compiler warnings in R-Tree code under Visual StudioExpress -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On

[sqlite] FTS Append?

2011-02-19 Thread Black, Michael (IS)
I think I already know the answer to this...but is it possible to append text to an FTS row without doing some sort of subselect? Even with the subselect what;s the best way to do this? I seem unable to find a single-liner that works. sqlite> create virtual table data using fts4(content te

Re: [sqlite] Is it possible to determine how many openconnections are active for a sqlite database?

2011-02-20 Thread Black, Michael (IS)
I don't know if it works for your data...but you don't need to do all 5.4M in one batch. You should test doing it in different batch sizes -- like 1000 at a time (and let other processes do their thing potentially). That way you won't lock them out. But I think your other selects need to use t

Re: [sqlite] EXT :Re: FTS Append?

2011-02-20 Thread Black, Michael (IS)
m: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Puneet Kishor [punk.k...@gmail.com] Sent: Saturday, February 19, 2011 10:21 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] FTS Append? On Saturday, February 19, 2011 at 10:17 AM, Black, Michael

Re: [sqlite] EXT :Re: sqlite3_busy_handler

2011-02-21 Thread Black, Michael (IS)
If I'm not mistaken only WAL mode supports simulaneous read/write. For any other mode any write function will lock the database. So...selects may run into a need to wait until a write finishes. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate

Re: [sqlite] sqlite3_busy_handler

2011-02-22 Thread Black, Michael (IS)
I dont' know the details of the busy handler. Not clear to me that it should sequentialize the requests. Perhaps you're better off just using a flag that you could check between your commit;begin so that if there's a request in the queue you go process it before continuing. commit; if item_in

Re: [sqlite] Parallel execution of queries in threads

2011-02-24 Thread Black, Michael (IS)
You don't say how many cores you have but I assume more than 4. If you're just single or dual-core I'd say this isn't too surprising. It's pretty rare that multi-threading gives an N*X performance boost -- especially for CPU or disk bound processes. Simon said most of the reasons but also for

Re: [sqlite] long insert statement failing on iPhone

2011-02-26 Thread Black, Michael (IS)
My first (and only) reaction is "bug in your code" overrunning a char buffer. If you could show where you do this maybe we can help. Problem is that it could be another statement doing the overrun too so as much code as you could share would help. Michael D. Black Senior Scientist NG Informati

Re: [sqlite] long insert statement failing on iPhone

2011-02-26 Thread Black, Michael (IS)
eptual/Multithreading/CreatingThreads/CreatingThreads.html Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Black, Michael (IS) [micha

Re: [sqlite] long insert statement failing on iPhone

2011-02-28 Thread Black, Michael (IS)
Did you increase the main thread size too? I would bump them both up by a LOT. A 2X change might not be enough. The simulator might grow the stack in a different direction so it's just not visible. Can you turn on stack checking in the compiler? Or stack-usage? If it's not the stack the

Re: [sqlite] EXT :Re: COUNT() extremely slow first time!

2011-02-28 Thread Black, Michael (IS)
Those numbers make sense to me. Since count is doing a linear walk throughcorrect? #1 SSDs more than likely don't do "read ahead" like a disk drive. So what you're seeing is what read-ahead does for you. #2 Count is doing a linear scan of the table...Probability of 2K containing the next p

Re: [sqlite] "select disctict" calles tcl functions twice?

2011-02-28 Thread Black, Michael (IS)
I think your problem may be that fun() in your eval is being called both from tcl and from sqlite. Name your proc fun2 and the problem will probably go away. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From:

Re: [sqlite] COUNT() extremely slow first time!

2011-02-28 Thread Black, Michael (IS)
w 1024), as you see the difference is only x2. I have only one explanation looking at the numbers. Although it's well-known that sqlite reads only full pages, if it sometimes does partial reading, this 5MB/Sec drop for <256 reading can affect linear speed of 25 MB/Sec to end up as 12MB/Sec. Bu

Re: [sqlite] EXT :Re: long insert statement failing on iPhone

2011-02-28 Thread Black, Michael (IS)
Well that's one you never mentioned...it works on the version with the SDK? What version of sqlite comes with that? And why are you upgrading sqlite? Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-u

[sqlite] :Re: long insert statement failing on iPhone

2011-02-28 Thread Black, Michael (IS)
My experience says the problem/solution is probably simple...FINDING it is the hard part. You'll likely change one or two lines of code unless it's a problem with stack usage on SQlite that has changed. Anybody done any measurement on SQLite stack usage? Michael D. Black Senior Scientist NG Inf

Re: [sqlite] EXT :Re: long insert statement failing on iPhone

2011-02-28 Thread Black, Michael (IS)
No such thing as "close enough" when it comes to different versions. Can somebody find the 3.6.23.2 amalgamation for this guy? Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org

Re: [sqlite] EXT :Re: long insert statement failing on iPhone

2011-03-01 Thread Black, Michael (IS)
Have you run your test with and without crypto? If Apple can compile 3.6.23.2 to work you should be able to also (might be overly optimistic here but compilers are 100% deterministic, although not necessartiliy 100% binary match). Michael D. Black Senior Scientist NG Information Systems Advance

Re: [sqlite] sqlite3_prepare_v2 and std::string using SQLite 3.7.5 - BUG or me?

2011-03-01 Thread Black, Michael (IS)
Try using SQLITE_TRANSIENT instead of STATIC. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of pcun...@fsmail.net [pcun...@fsma

Re: [sqlite] Adjusting strategy for ROWIDs

2011-03-04 Thread Black, Michael (IS)
SQlite's random() is a pseudo-random (as are most all) so there is no collision until you get the same value back at which point it just repeats the whole sequence again. So the following example should work fine for him. When it collides you've cycled through the complete range of SQlite's ra

Re: [sqlite] Adjusting strategy for ROWIDs

2011-03-05 Thread Black, Michael (IS)
I wasn't aware SQLite's PRNG was not like most others. Good to know. I guess one could insert their own random() function if you need repeatability (which is actually a major point of most random number generators). I don't see the seeding exposed in SQLite so you can restart from a given point

[sqlite] Documentation bug?

2011-03-06 Thread Black, Michael (IS)
I was trying to follow the documentation on FTS4 and found what I guess is a typo. In several places there is a reference to "document" where I believe it should be "documents" as the first term doesn't exist in any create statement. There are several of these in Appendix A http://www.sqlite.or

Re: [sqlite] Indexing - a test example

2011-03-07 Thread Black, Michael (IS)
You should learn how to use "explain query plan". Indexes are a trade-off between insert speed (slow) and select speed (fast). If you have a static database there is no such thing as too many indexes as they never would get changed. Most of us live in the world between. Generally, don't worry

Re: [sqlite] sqlite DB extraction

2011-03-08 Thread Black, Michael (IS)
You go from 2 minutes to 30 minutes just based on clock speed. I would imagine you don't have much disk cache on the ARM device, so every time you read the database it has to hit the file systemouch Did you benchmark your PC system without using a cached database? Just edit the database

Re: [sqlite] Optimising a query with several criteria

2011-03-13 Thread Black, Michael (IS)
You don't say how big your database is. My guess is when you see the server using a lot of RAM (and exactly how are you measuring this?) that it's flushing its disk cache. If you're on Unix use vmstat to see what your OS cache is doing. So...perhaps if you increase SQLite's internal cache it

Re: [sqlite] Optimising a query with several criteria

2011-03-13 Thread Black, Michael (IS)
mroth.com] Sent: Sunday, March 13, 2011 8:40 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] Optimising a query with several criteria Hi Michael, thanks for this. My database is 40 megabytes (and growing slowly) - is that a reasonable cachesize? On 13/03/2011 13:07, Bla

Re: [sqlite] Optimization SQLite

2011-03-14 Thread Black, Michael (IS)
Try CodeBlocks Cross-platform and works with gcc or MSVC or pretty much whatever. I put in gcc 64-bit for it. http://www.codeblocks.org/ Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@

Re: [sqlite] Newbie question

2011-03-21 Thread Black, Michael (IS)
Try this. BEGIN TRANSACTION; CREATE TABLE sales(SaleDate date,SaleVolume int); INSERT INTO "sales" VALUES('2010-01-01', 10); INSERT INTO "sales" VALUES('2010-01-02', 20); INSERT INTO "sales" VALUES('2011-01-01', 15); INSERT INTO "sales" VALUES('2011-01-02', 30); INSERT INTO "sales" VALUES('2009-01

Re: [sqlite] Bulk loading large dataset; looking for speedups.

2011-03-22 Thread Black, Michael (IS)
But I thought he said he dropped the indexes (meaning they aren't there during inserts). That should make sorting irrelevant. 3 Things. #1 Test with :memory: database and see what the speed is. That tells you if it's SQLite or disk I/O as the bottleneck. #2 Try WAL mode "pragma journal_mode=WA

Re: [sqlite] storing big numbers into NUMERIC, DECIMAL columns

2011-03-22 Thread Black, Michael (IS)
I hope you know what you're doing with trying to preserve that much significance. Ths first time you stick it in a double or long double variable you'll lose it. You can use the HPAlib to get 32 digits http://www.nongnu.org/hpalib/ // Example showing digit loss -- doesn't matter double or lo

Re: [sqlite] storing big numbers into NUMERIC, DECIMAL columns

2011-03-22 Thread Black, Michael (IS)
Hmmm...the docs do say that...but how do you get that value back out? Retreiving it as text doesn't work. You still don't say what you're planning on doing with these number...just displaying them? I think the docs may be misleading...here is the comment in sqlite3.c /* ** Try to convert a valu

Re: [sqlite] storing big numbers into NUMERIC, DECIMAL columns

2011-03-23 Thread Black, Michael (IS)
Blob may be better if you need speed -- then no conversion is necessary inside your Pascal code to/from a string. But if you want to be able to see and understand your database text is better (or you have to write a special Pascal program to decode your database to look at any problems). And...

Re: [sqlite] Reg: Link error while using sqlite 3

2011-03-23 Thread Black, Michael (IS)
Unless you're running multiple SQLite apps you don't gain anything by using a DLL. Plus, is your Pocket PC a i386 CPU? So try downloading the amalgamation and include sqlite3.c and sqlite3.h in your project. http://www.sqlite.org/sqlite-amalgamation-3070500.zip You'll also find the code will be

Re: [sqlite] storing big numbers into NUMERIC, DECIMAL columns

2011-03-23 Thread Black, Michael (IS)
Hmmm...according to my math... Max 64-bit unsigned integer is 18446744073709551615 Drop the last digit as it can't hold 0-9 1844674407370955161 Make two decimal positions 18446744073709551.61 Now some commas so we can see better 18,446,744,073,709,551.61 That' $18 quadrillion dollars by my math.

Re: [sqlite] Full Table Scan after Analyze

2011-03-26 Thread Black, Michael (IS)
When you say "All an index does" don't forget that an index is also usually smaller than the data, thereby increase cache performance and reducing disk seeks. For a good chunk of typical uses (large tables with simple lookups) an index is notably faster. I'll admit my use of sqtlite3 hasn't

Re: [sqlite] Lack of "decimal" support

2011-03-27 Thread Black, Michael (IS)
Base 10 multiplication is needed. Although money is the main reason for doing BCD due to cumulative errors it's not the only reason. So I'd recommend adding decimal*decimal -> decimal just to be complete. It's easy enough to implement using the + function so could just be noted as "slow". Th

Re: [sqlite] sqlite3_step behavior on empty set

2011-03-28 Thread Black, Michael (IS)
Wouldn't the addition of "Empty result sets will return SQLITE_DONE on the first call to sqlite3_step." add some clarity? Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqli

Re: [sqlite] bug with sqlite

2011-03-29 Thread Black, Michael (IS)
Just to help clarify (hopefully) the Unix/Windows "reserved filename". CON: is similar to Unix's /dev/zero or /dev/null for example -- Files that already exist and have OS meaning. stdout is NOT a reserved filename...it's a predefined variable of FILE *. You cannot say "cp file stdout" on Unix

Re: [sqlite] Generate a unique id unless it is provided

2011-03-29 Thread Black, Michael (IS)
I don't understand how your column C works...so I'll assume it's pre-known for now. But here's how to get A/B to work. drop table t if exists; create table t(a int,b int,c int); create trigger trig1 after insert on t begin update t set a=(select count(b) from t where b=new.b) where a=0; end; ins

Re: [sqlite] mutex assert_fail in btreeInvokeBusyHandler occasionally in a periodic DB update in 3.5.7, It's ok in 3.6.22(-DSQLITE_THREADSAFE=1)

2011-03-31 Thread Black, Michael (IS)
Since it apperas you're running your commit in a separate thread and are therefore muilti-threaded I do belive you need: SQLITE3_THREADSAFE=2 >From http://www.sqlite.org/compile.html#threadsafe To put it another way, SQLITE_THREADSAFE=1 sets the default threading mode to Serialized. SQLITE_THREA

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char

2011-04-03 Thread Black, Michael (IS)
You apparently don't understand "strings" in C. Or are you actually reading in binary data? #1 Since you said "image" I assume you're reading binaary. So get rid of buffer[fsize]=0. You don't null terminate binary data and that statement is 1-beyond the end of the array (which is from 0 to fs

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char

2011-04-03 Thread Black, Michael (IS)
How are you trying to view the ouitput.result.txt (and I"ll note that it'sNOT a text file...it's an image according to what you said.). What's the size of the file. And you should be able to post a COMPLETE example to show your testing. What you say you want to do has been done by many before

Re: [sqlite] sqlite3_bind_blob CHOPS off at first NULL char - FULL CODE

2011-04-03 Thread Black, Michael (IS)
I ran your code with my test file and I get this...which is perfectly correct. Do you get something different? What makes you think the stream is truncated in the database? Also...change SQLITE_STATIC to SQLITE_TRANSIENT...that could be your culprit if you are still seeing truncation. Plus "

[sqlite] :Re: sqlite3_bind_blob CHOPS off at first NULL char

2011-04-03 Thread Black, Michael (IS)
A B nl 003 I must be doing something fundamentally wrong.;-( Thanks for the help so far, I really appreciate it. Lynton On 03/04/2011 13:44, Black, Michael (IS) wrote: > How are you trying to view the ouitput.result.txt (and I"ll note that it'sNOT > a text file..

Re: [sqlite] sqlite-amalgamation + own functions = Segmentation fault at "sqlite3VdbeMemSetStr" sice version 3.7.5 (to v3.7.3 works fine)

2011-04-16 Thread Black, Michael (IS)
Care to show us your SPL_mallocstr() function? Sounds like you've corrupted data if that line dies. All it's doing is checking an array value which is used all over the place in sqlite3.c If you put a break point there and on the first time it's hit put a watch on the address for u.ao.nByte>db

Re: [sqlite] 'integer'

2011-04-17 Thread Black, Michael (IS)
Seems to behave OK for me on 3.7.5 on Windows. What version are you using on what OS with what compile flags? You also "said" it didn't work but you didnt' actually what what you did. Like this... SQLite version 3.7.5 Enter ".help" for instructions Enter SQL statements terminated with a ";"

Re: [sqlite] 'integer'

2011-04-18 Thread Black, Michael (IS)
That's the nice thing about standards...there's so many to choose from :-) Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Simon Sl

Re: [sqlite] What happens if you insert more than your RAM size into an in memory database?

2011-04-18 Thread Black, Michael (IS)
Ummm...are we forgetting about swap space? If you exceed RAM you hit swap. If you exceed RAM+SWAP you start failing. Or does sqlite monitor physical memory usage? So if you exceed RAM you just start slowing down towards a disk-based equialent database. Michael D. Black Senior Scienti

Re: [sqlite] Request for help with SQLite Query to return missing Date/Time Ranges

2011-04-21 Thread Black, Michael (IS)
Assuming your database ONLY contains the log entries this should workand be pretty fast too since rowid is already indexed and there areYou no other lookups. You can add your own rowid to make this work otherwise. Just do a max(myrowid)+1 on your insert. PRAGMA foreign_keys=OFF; BEGIN

Re: [sqlite] date field with default current date

2011-04-21 Thread Black, Michael (IS)
create table t (d default CURRENT_DATE,i number); insert into t (i) values(1); select * from t; 2011-04-21|1 Use CURRENT_TIME if you want hours/minutes too. Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sql

Re: [sqlite] Request for help with SQLite Query to return missing Date/Time Ranges

2011-04-21 Thread Black, Michael (IS)
A slight mod on my solution makes it work for DST changes too. Again...rowid must be maintained. PRAGMA foreign_keys=OFF; BEGIN TRANSACTION; CREATE TABLE log(d date); INSERT INTO "log" VALUES('2011-03-13 01:55'); INSERT INTO "log" VALUES('2011-03-13 01:56'); INSERT INTO "log" VALUES('2011-03-1

Re: [sqlite] EXT :Re: BUG : round(x,y) not consistent

2011-04-30 Thread Black, Michael (IS)
3.75 on Redhat 5.6 does not have a problem. I get 8.88 16.88 32.88 64.88 Fixed in 3.7.5 maybe? Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sq

[sqlite] RE : Feature request: Fuzzy searching

2011-05-03 Thread Black, Michael (IS)
Sounds like soundex might work for you. Not sure how it handles diacritics though. http://www.mail-archive.com/sqlite-users@sqlite.org/msg35316.html Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-us

Re: [sqlite] EXT : Transaction speed too slow?

2011-05-09 Thread Black, Michael (IS)
Michael D. Black Senior Scientist NG Information Systems Advanced Analytics Directorate From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of Nick [maill...@css-uk.net] Sent: Sunday, May 08, 2011 3:52 PM To: sqlite-users@sql

Re: [sqlite] Transaction speed too slow?

2011-05-09 Thread Black, Michael (IS)
You do realize the number they quote is a MAXnot necessarily what you'll get. With 16 transactions per second you're writing on transaction every 63ms. My understanding is that the hard drive in the netbook is 15ms access time so you're seeing 4 accesses per transaction with the way you'

Re: [sqlite] Memory leak in SQlite

2011-05-10 Thread Black, Michael (IS)
#1 I don't see where you're freeing m_szErrorString (not real sure if it gets malloc'd on success) -- but you do need to free it on errors for sure. And where's your Callback function? Why are you calling SaveResultSet (which you also don't show)? That should probably be done inside the Call

Re: [sqlite] Blob newbie problem

2011-05-12 Thread Black, Michael (IS)
Do I understand you're still seeing a segfault? I assume you're not seeing your "Year retrieved..." statement? You haven't showed us your table definition. Change your strcmp to strcasecmp and see if that fixes it for you. Michael D. Black Senior Scientist NG Information Systems Adv

Re: [sqlite] disk image malformed

2012-12-06 Thread Black, Michael (IS)
Can you try doing table copies using select with offset and limit? See where limit crashes the system and then offset past it perhaps? Just a guess on my part as one possibility. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Gr

Re: [sqlite] Simple SQLite-based spreadsheet?

2012-12-07 Thread Black, Michael (IS)
http://www.sqlmaestro.com May do what you want... Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sql

Re: [sqlite] sqlite3 windows command line: writing the sql errors to a file (and not to the screen)

2012-02-14 Thread Black, Michael (IS)
type dbscript.sql | sqlite3 test.db3 > dbscript.log 2>&1 And order is importantyou need to redirect to file first and then redirect stderr to stdout as above. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman

[sqlite] RE SELECT average timestamp to get average time of day?

2012-02-25 Thread Black, Michael (IS)
I subtract 12 hours...so any time from 24:00:00 to 12:00:00 will work. Time from noon to noon becomes midnight to midnight. Then you just add the 12 hours back in. CREATE TABLE tijd(t int(11)); INSERT INTO "tijd" VALUES('2012-02-25 22:00:00'); INSERT INTO "tijd" VALUES('2012-02-27 01:00:00')

Re: [sqlite] TEXT PRIMARY KEY

2012-03-02 Thread Black, Michael (IS)
Hmmm...works for me... On Windows: SQLite version 3.7.9 2011-11-01 00:52:41 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> create table test(a text primary key); sqlite> insert into test values('1'); sqlite> insert into test values('1'); Error: column a is not

Re: [sqlite] Efficient random sampling in a large table using builtin functions.

2012-03-08 Thread Black, Michael (IS)
You don't say what language you are working in. IN C++ I would just declare a "set" and put random row numbers in it until I had enough. Then use that set to build the SQL. SQLite's random() doesn't have a seed function so you don't really get very random numbers from run-to-run and have no

Re: [sqlite] Efficient random sampling in a large table using builtin functions.

2012-03-08 Thread Black, Michael (IS)
] Efficient random sampling in a large table using builtin functions. On 8 March 2012 14:20, Black, Michael (IS) wrote: > You don't say what language you are working in. IN C++ I would just declare > a "set" and put random row numbers in it until I had enough. Then use that &

Re: [sqlite] Efficient random sampling in a large table using builtin functions.

2012-03-08 Thread Black, Michael (IS)
avies [simon.james.dav...@gmail.com] Sent: Thursday, March 08, 2012 8:47 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] Efficient random sampling in a large table using builtin functions. On 8 March 2012 14:37, Black, Michael (IS) wrote: > Glad to know thatcould that pos

Re: [sqlite] SIGBUS error in case of disk full with WAL mode

2012-03-08 Thread Black, Michael (IS)
Looks like this should work... >From http://www.cs.cf.ac.uk/Dave/C/node27.html The following code fragment demonstrates a use of this to create a block of scratch storage in a program, at an address that the system chooses.: int fd; caddr_t result; if ((fd = open("/dev/zero", O_RDWR)) == -1

Re: [sqlite] SIGBUS error in case of disk full with WAL mode

2012-03-08 Thread Black, Michael (IS)
at 11:45 AM, Black, Michael (IS) wrote: > Looks like this should work... > No, it won't work. The memory has to be shared in common among all connections to a particular database. If two separate processes connection to the same database, they must get the same block of shared memory.

Re: [sqlite] EXT : Sqlite error message

2012-03-15 Thread Black, Michael (IS)
Put an absolute path name for your filename. You're probably opening up the database in the wrong directory, creating an empty DB, and thus "no such table". In particular this happens when running from the IDE or via an Icon where the working directory is not set. Michael D. Black Senior

Re: [sqlite] Optimize Table Access Efficiency

2012-03-20 Thread Black, Michael (IS)
Try creating 2 tables, one for topics, one for definitions. Then insert all the topics at once followed by all the definitions. That should give you the same disk layout as two databases. And you don't say what "lengthy" means. Michael D. Black Senior Scientist Advanced Analytics Direct

Re: [sqlite] Optimize Table Access Efficiency

2012-03-20 Thread Black, Michael (IS)
sh start. Tim On 3/20/2012 10:26 AM, Black, Michael (IS) wrote: > Try creating 2 tables, one for topics, one for definitions. > > > > Then insert all the topics at once followed by all the definitions. > > That should give you the same disk layout as two databa

Re: [sqlite] 64-bit Windows Command Shell

2012-03-21 Thread Black, Michael (IS)
Cache is the primary (and obvious) thing I can think of. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems From: sqlite-users-boun...@sqlite.org [sqlite-users-boun..

Re: [sqlite] 64-bit Windows Command Shell

2012-03-26 Thread Black, Michael (IS)
n pretty much expect. Would getting a server with 4 CPUs and 16GB (a > > high-end home-version PC) - reasonably enable me to run 3-4 SQLite jobs > > concurrently? In other words - no great speed improvement per job - but > in > > aggregate more work could get done? > > >

Re: [sqlite] memory handling problems in 3.710 - 3.7.11

2012-03-27 Thread Black, Michael (IS)
Don't see any problems here with valgrind. Red Hat Enterprise Linux Server release 5.7 (Tikanga) [sqlite-amalgamation-3071100]$ gcc -g -o shell shell.c sqlite3.c -ldl -lpthread [sqlite-amalgamation-3071100]$ ./shell SQLite version 3.7.11 2012-03-20 11:35:50 Enter ".help" for instructions Ente

Re: [sqlite] how to add time in a new column

2012-03-29 Thread Black, Michael (IS)
You'll need to export the table and data. Change the SQL to what you want. Then import again. Does the shell have ability to name the columns on the insert statements from the .dump to make this easier? I don' t see anythinig offhand that seems to do that. Michael D. Black Senior Sci

Re: [sqlite] about sqlite3_exec function

2012-03-31 Thread Black, Michael (IS)
What you want is the system() function which will execute a shell command. But you still need to add your own HTML around it to be displayed by a browser as it's missing the "rest of the story". system("echo >mm.html"); // first one creates mm.html system("echo >>mm.html"); // 2nd and su

Re: [sqlite] about sqlite3_exec function

2012-03-31 Thread Black, Michael (IS)
ite.org] on behalf of Simon Slavin [slav...@bigfraud.org] Sent: Saturday, March 31, 2012 7:50 AM To: General Discussion of SQLite Database Subject: EXT :Re: [sqlite] about sqlite3_exec function On 31 Mar 2012, at 12:48pm, "Black, Michael (IS)" wrote: > What you want is the system() functi

Re: [sqlite] Hashing 2 SQLite db files with the same data

2012-04-03 Thread Black, Michael (IS)
Database files are purportedly platform independent. So why don't you distribute the database file instead of building it? Then your checksum would be fine. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Informati

Re: [sqlite] Hashing 2 SQLite db files with the same data

2012-04-04 Thread Black, Michael (IS)
Howeverthe DB file is portable across big/little endian and 32/64 bit. So do your hash on the DB file and distribute that. Any reason you can't do that? http://www.sqlite.org/onefile.html I guess SQLite uses the endianess of the database file over the architecture? Michael D. Blac

Re: [sqlite] Variation on INSERT with SELECT issue

2012-04-05 Thread Black, Michael (IS)
You need 2 inserts to do what you want. Hopefully the order in the table doesn't matter to you. sqlite> CREATE TABLE t2 (Col1 text,Col2 text); sqlite> insert into t2 (Col1) values('xxx'); sqlite> insert into t2 values('yyy','def'); sqlite> select * from t2; xxx| yyy|def sqlite> sqlite> sqlite>

Re: [sqlite] undefined symbol: sqlite3_stricmp

2012-04-08 Thread Black, Michael (IS)
Don't you need to load the sqlite3 library first for tclsh? load ./libtclsqlite3.so Sqlite3 Or something like that? Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems __

Re: [sqlite] case where sqlite3_prepare_v2 returns OK and sets stmt to NULL

2012-04-09 Thread Black, Michael (IS)
You want your assert to be this: assert(stmt != NULL); Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems From: sqlite-users-boun...@sqlite.org [sqlite-users-bou

Re: [sqlite] error compilation with sqlite amalgamation

2012-04-11 Thread Black, Michael (IS)
You don't show how you're compiling...but this is what you need to do. gcc -o myprog myprog.c sqlite3.c -lthread -ldl Your undefined reference are to the two libraries you need to link in to resolve them. Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOIN

Re: [sqlite] error compilation with sqlite amalgamation

2012-04-11 Thread Black, Michael (IS)
'origine- De : sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] De la part de Black, Michael (IS) Envoyé : mercredi 11 avril 2012 14:01 À : General Discussion of SQLite Database Objet : Re: [sqlite] error compilation with sqlite amalgamation You don't show how y

Re: [sqlite] Using "attach database" to work around DB locking

2012-04-11 Thread Black, Michael (IS)
I assume you are batching your inserts? How many inserts/sec do you need to do to keep up? Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems From: sqlite-users-boun

Re: [sqlite] error compilation with Sqlite in C program

2012-04-12 Thread Black, Michael (IS)
You're missing a step in your library build. ranlib libsqlite.a Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems From: sqlite-users-boun...@sqlite.org [sqlite-

Re: [sqlite] error compilation with Sqlite in C program

2012-04-12 Thread Black, Michael (IS)
rs-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] De la part de Black, Michael (IS) Envoyé : jeudi 12 avril 2012 17:17 À : General Discussion of SQLite Database Objet : Re: [sqlite] error compilation with Sqlite in C program You're missing a step in your library build.

Re: [sqlite] Slightly unexpected behaviour when comparing date and datetime

2012-04-17 Thread Black, Michael (IS)
Store them as float or do integer and multiple by a power of 10 to get as many digits as you want. So 1.234 seconds *10^3 can be 1234 integer Michael D. Black Senior Scientist Advanced Analytics Directorate Advanced GEOINT Solutions Operating Unit Northrop Grumman Information Systems

Re: [sqlite] Time zones

2012-04-17 Thread Black, Michael (IS)
You're talking PHP...not SQLite. SQLite doesn't know about timezones other than "local" and "utc". So your timezones will depend on your OS. On RedHat it's in /usr/share/zoneinfo and there's tons of them. I've got 1,743 of them. Michael D. Black Senior Scientist Advanced Analyti

Re: [sqlite] "DEFAULT BOOLEAN NOT NULL" not working with entityframework

2012-04-18 Thread Black, Michael (IS)
Have you checked your table afterwords to ensure you don't have any nulls in IsReplaced? select count(IsReplaced) from mytable where IsReplaced is null; I tested and the alter table does fill with default values for me. At least from the sqlite shell. Does this work for you? Are you doin

Re: [sqlite] "DEFAULT BOOLEAN NOT NULL" not working with entityframework

2012-04-18 Thread Black, Michael (IS)
default values are during INSERT...not SELECT. I suppose it's possible Maestro is messing it up. You just need to do an "update mytable set IsReplaced=0 where IsReplaced is null;" You can do that from the shell. Hopefully that makes your ADO happy. If you want to test the shell then d

<    1   2   3   4   5   6   7   8   9   >