Re: [sqlite] Is there any API for counting number of rows in a particular table.

2012-01-31 Thread John Elrick
On Tue, Jan 31, 2012 at 6:01 AM, Fabio Spadaro wrote: > Hi > > 2012/1/30 Bart Smissaert > > > How do you make it open database files that have an extension other than > > .slt? > > > > RBS > > > Tonight I will see to create a new version with

Re: [sqlite] Found it

2012-01-26 Thread John Elrick
On Thu, Jan 26, 2012 at 12:36 PM, Stephan Beal wrote: > On Thu, Jan 26, 2012 at 2:53 PM, Mohit Sindhwani wrote: > > > Absolutely! I come home from work and tune in to this thread, gripped :) > > > > +1 to Mohit and the others who's written similar

Re: [sqlite] Found it

2012-01-26 Thread John Elrick
On Thu, Jan 26, 2012 at 3:29 AM, Jean-Denis MUYS <jdm...@kleegroup.com>wrote: > > On 25 janv. 2012, at 22:30, John Elrick wrote: > > > > > I have to say this has been a fascinating learning experience. It has > been > > enlightening reading the SQLite code and

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Wed, Jan 25, 2012 at 4:21 PM, Nico Williams wrote: > On Wed, Jan 25, 2012 at 3:07 PM, Richard Hipp wrote: > > The SQLite byte-code engine was being too conservative and was reparsing > > the schema in places where it was not strictly necessary. The fix

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Wed, Jan 25, 2012 at 4:07 PM, Richard Hipp <d...@sqlite.org> wrote: > On Wed, Jan 25, 2012 at 4:01 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > Much better. I wasn't sure which version to incorporate the changes into > > (the link is to the

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
Thanks. On Wed, Jan 25, 2012 at 4:11 PM, Petite Abeille <petite.abei...@gmail.com>wrote: > > On Jan 25, 2012, at 10:09 PM, John Elrick wrote: > > > DML? > > http://en.wikipedia.org/wiki/Data_Manipulation_Language > > > ___

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Wed, Jan 25, 2012 at 4:07 PM, Richard Hipp <d...@sqlite.org> wrote: > On Wed, Jan 25, 2012 at 4:01 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > Much better. I wasn't sure which version to incorporate the changes into > > (the link is to the

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- John Elrick Fenestra Technologies 540-868-1377 ___ sqlite-users mailin

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
Richard, I've found a problem that I don't understand. Looking at the case OP_VerifyCookie, I attempted to output pOp->opcode, which I expected to equal OP_VerifyCookie. However, it doesn't: Schema has changed current state = 'db=0025, pOp->opCode=66785672,

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
Richard, Is it reasonable to say that, assuming: 1. The schema has not changed 2. There is only one connection to the database that the value of iGeneration should always equal pOp->p3 in OP_VerifyCookie? ___ sqlite-users mailing list

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Wed, Jan 25, 2012 at 1:13 PM, Richard Hipp <d...@sqlite.org> wrote: > On Wed, Jan 25, 2012 at 10:18 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > I do have some more information for you. I tracking everything which > could &g

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
Addendum On Wed, Jan 25, 2012 at 1:12 PM, John Elrick <john.elr...@fenestra.com>wrote: > On Wed, Jan 25, 2012 at 9:50 AM, Richard Hipp <d...@sqlite.org> wrote: > >> On Wed, Jan 25, 2012 at 9:02 AM, John Elrick <john.elr...@fenestra.com >> >wrote: >>

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Wed, Jan 25, 2012 at 9:50 AM, Richard Hipp <d...@sqlite.org> wrote: > On Wed, Jan 25, 2012 at 9:02 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > current state = 'db=03FB1188, db->aDb[pOp->p1].pSchema=02110AB0, > > db->aDb[pOp->

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
Ok. Possible scenario. I can't prove it, but somebody tell me if this is what could be going on: prepare query1 prepare query2 create something -> schema change query1.step - schema invalid so recreate - recreation of schema changes the generation query2.step - schema invalid so recreate

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Wed, Jan 25, 2012 at 10:32 AM, Simon Slavin <slav...@bigfraud.org> wrote: > > On 25 Jan 2012, at 3:18pm, John Elrick wrote: > > > So, it appears that something is making SQLite think the schema has > changed. > > Can you read the following: > >

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Wed, Jan 25, 2012 at 9:50 AM, Richard Hipp <d...@sqlite.org> wrote: > On Wed, Jan 25, 2012 at 9:02 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > current state = 'db=03FB1188, db->aDb[pOp->p1].pSchema=02110AB0, > > db->aDb[pOp->

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
ver invoked. Whatever is resetting the internal schema isn't coming from here. On Wed, Jan 25, 2012 at 9:02 AM, John Elrick <john.elr...@fenestra.com>wrote: > > On Tue, Jan 24, 2012 at 7:46 PM, Richard Hipp <d...@sqlite.org> wrote: > >> On Tue, Jan 24, 2012 at 7:37 PM

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Tue, Jan 24, 2012 at 7:46 PM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 24, 2012 at 7:37 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > It appears that the value is set here: > > > > u.av.pBt = db->aDb[pOp->p1].p

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
On Wed, Jan 25, 2012 at 8:02 AM, Richard Hipp <d...@sqlite.org> wrote: > On Wed, Jan 25, 2012 at 7:53 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > I started with the place where iGeneration was being changed and found > > nothing out of the or

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
...especially given that the results of sqlite3SchemaClear are being passed as a parameter and from the declaration, sqlite3SchemaClear doesn't return anything. On Wed, Jan 25, 2012 at 7:53 AM, John Elrick <john.elr...@fenestra.com>wrote: > I started with the place where iGeneration

Re: [sqlite] Found it

2012-01-25 Thread John Elrick
ashInit(>tblHash); sqlite3HashInit(>idxHash); sqlite3HashInit(>trigHash); sqlite3HashInit(>fkeyHash); p->enc = SQLITE_UTF8; } return p; } On Tue, Jan 24, 2012 at 7:46 PM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 24, 2012 at 7:37 PM, John E

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 7:26 PM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 24, 2012 at 7:18 PM, John Elrick <john.elr...@fenestra.com > >wrote: > SNIP > So the problem was introduced by > > http://www.sqlite.org/src/info/36c04dd1695f08 >

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
I really found it this time. While I was looking over the code changes, I noticed a section of code dealing with case OP_VerifyCookie: Inside that section is the following in 3.7.6 if( u.av.iMeta!=pOp->p2 || u.av.iGen!=pOp->p3 ){ sqlite3DbFree(db, p->zErrMsg); p->zErrMsg =

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
Let me rephrase that... I ran this test with a small case. See results below. On Tue, Jan 24, 2012 at 5:55 PM, John Elrick <john.elr...@fenestra.com>wrote: SNIP > > http://www.sqlite.org/fenstra/sqlite-201102040051.zip >> > > Low malloc count > 8,191 calls to ma

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 5:36 PM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 24, 2012 at 5:21 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > On Tue, Jan 24, 2012 at 3:05 PM, Richard Hipp <d...@sqlite.org> wrote: > > > > &g

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 3:05 PM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 24, 2012 at 1:28 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > > > > http://www.sqlite.org/fenstra/sqlite-201103081639.zip > > > > > >

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
Nope and nope. They are happening apparently out of the blue. On Jan 24, 2012 2:40 PM, "Nico Williams" wrote: > Maybe a view is getting materialized, or a some other temp table's > creation under the hood is triggering this? >

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 12:53 PM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 24, 2012 at 12:15 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > > > Can you put a breakpoint on sqlite3ResetInternalSchema() and figure out > &

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 1:28 PM, John Elrick <john.elr...@fenestra.com>wrote: SNIP http://www.sqlite.org/fenstra/sqlite-201103241737.zip > > > This one is failing to compile. It needs _msize. That sounds like the > 3.7.10 issue reported recently. > > I o

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 1:29 PM, Krystian Bigaj wrote: > You can compile sqlite3.dll with debug information, and then you can in VS > attach to your running .exe - you should be able to see sqlite > stacks/breakpoints/step-in/out/etc. should also work. > > Thanks very

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 12:53 PM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 24, 2012 at 12:15 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > SNIP > http://www.sqlite.org/fenstra/sqlite-201103081639.zip This one gives the high number of mallocs jus

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 1:18 PM, Dan Kennedy <danielk1...@gmail.com> wrote: > On 01/25/2012 12:15 AM, John Elrick wrote: > >> On Tue, Jan 24, 2012 at 12:06 PM, Richard Hipp<d...@sqlite.org> wrote: >> >> On Tue, Jan 24, 2012 at 11:59 AM, John >>

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
On Tue, Jan 24, 2012 at 12:06 PM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 24, 2012 at 11:59 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > While reviewing our other thread, I noticed a piece of information > > concerning sqlite3ResetInt

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
While reviewing our other thread, I noticed a piece of information concerning sqlite3ResetInternalSchema. Based on the number of times that I reported it had been called, I believe that something was changed between 3.7.5 and 3.7.6 which causes sqlite3ResetInternalSchema to be called when it

Re: [sqlite] Found it

2012-01-24 Thread John Elrick
Addendum. After looking further it appears that the reparsing is happening independently of triggering. In this case, the reparsing occurs 30,502 times for every view and trigger I have examined. Only one trigger appears in the 3.7.5 listing and it is parsed exactly 1 time. 3.7.5 CREATE

[sqlite] Found it

2012-01-24 Thread John Elrick
Richard, Your last request for some detailed information about what queries were being processed did the trick. Here is a description of the problem (and yes, it appears to be in SQLite). In 3.7.5, the number one malloc consumer is INSERT INTO SEQUENCE_ELEMENTS ( SEQUENCE_ELEMENT_OID,

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-23 Thread John Elrick
On Mon, Jan 23, 2012 at 4:27 PM, Nico Williams <n...@cryptonector.com>wrote: > On Mon, Jan 23, 2012 at 3:02 PM, John Elrick <john.elr...@fenestra.com> > wrote: > > I think I can inject something to do some measurements. I seem to > recall, > > however, that th

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-23 Thread John Elrick
On Mon, Jan 23, 2012 at 3:48 PM, Richard Hipp <d...@sqlite.org> wrote: > On Mon, Jan 23, 2012 at 3:12 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > Brain hurts... > > > > Richard, more information for you. I rebuilt the call stack checking &g

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-23 Thread John Elrick
Brain hurts... Richard, more information for you. I rebuilt the call stack checking system into pure Delphi and confirmed that yy_reduce appears to be the malloc culprit. I further created a pair of procedures which I can use to track the yyruleno from yy_reduce as though it were a call. Below

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-22 Thread John Elrick
ault is in the Delphi bridge to SQLite and any > pure examination of SQLite's behaviour isn't going to spot anything, right > ? So a better place to query this would be a Delphi list ? > > Simon. > ___ > sqlite-users mailing list > sq

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-21 Thread John Elrick
. As frustrating as it has been to narrow down the cause, I have nothing but respect and gratitude for everyone who has attempted to help. Thank you all. On Jan 20, 2012 3:47 PM, "Richard Hipp" <d...@sqlite.org> wrote: > On Fri, Jan 20, 2012 at 3:40 PM, John Elrick <john.elr...

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-20 Thread John Elrick
20, 2012 at 2:55 PM, John Elrick <john.elr...@fenestra.com>wrote: > The problem is not in the queries. The problem is in a two order of > magnitude increase in _mallocs between the versions. The _mallocs are > coming from sqlite3Parser. > > > On Fri, Jan 20, 2012 at 2:37

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-20 Thread John Elrick
The problem is not in the queries. The problem is in a two order of magnitude increase in _mallocs between the versions. The _mallocs are coming from sqlite3Parser. On Fri, Jan 20, 2012 at 2:37 PM, Max Vlasov <max.vla...@gmail.com> wrote: > On Fri, Jan 20, 2012 at 10:05 PM, Jo

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-20 Thread John Elrick
The change which results in a slow down occurred between 3.7.5.0 and 3.7.6.0. On Fri, Jan 20, 2012 at 8:34 AM, John Elrick <john.elr...@fenestra.com>wrote: > Thank you sir. I'll start looking. > > > On Fri, Jan 20, 2012 at 8:27 AM, Richard Hipp <d...@sqlite.org> wrote: &

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-20 Thread John Elrick
Thank you sir. I'll start looking. On Fri, Jan 20, 2012 at 8:27 AM, Richard Hipp <d...@sqlite.org> wrote: > On Fri, Jan 20, 2012 at 8:14 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > I asked in an earlier posting if the amalgamations were available >

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-20 Thread John Elrick
> both good and bad EXPLAIN QUERY output here. I'm sure this will be greek > for most of us :), but when Richard or Dan look at the difference in the > vdbe code, they could notice something > > Max > > On Fri, Jan 20, 2012 at 1:17 AM, John Elrick <john.elr...@fen

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-19 Thread John Elrick
One more useful comparison. The following query is prepared and step is called with these results: step: UPDATE RESPONSES SET RESPONSE_NAME = :RESPONSE_NAME WHERE RESPONSE_OID = :RESPONSE_OID Cumulative Allocated Memory: 10,992 Count of _malloc Calls:18

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-19 Thread John Elrick
On Thu, Jan 19, 2012 at 11:49 AM, John Elrick <john.elr...@fenestra.com>wrote: > I've added more tracking to our libraries. Most queries result in minimal > (<200 calls) _malloc activity, But I have found several anomalies. I am > listing some representatives below. In

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-19 Thread John Elrick
I've added more tracking to our libraries. Most queries result in minimal (<200 calls) _malloc activity, But I have found several anomalies. I am listing some representatives below. In the examples below, "step" is the operation. Note also that all queries should have been prepared with

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-19 Thread John Elrick
On Thu, Jan 19, 2012 at 8:25 AM, John Elrick <john.elr...@fenestra.com>wrote: > > > On Thu, Jan 19, 2012 at 8:15 AM, Richard Hipp <d...@sqlite.org> wrote: > >> On Thu, Jan 19, 2012 at 8:10 AM, John Elrick <john.elr...@fenestra.com >> >wrote: >> >

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-19 Thread John Elrick
On Thu, Jan 19, 2012 at 8:15 AM, Richard Hipp <d...@sqlite.org> wrote: > On Thu, Jan 19, 2012 at 8:10 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > I've been reading through the code. Do I understand correctly that if > one > > prepared

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-19 Thread John Elrick
On Thu, Jan 19, 2012 at 8:03 AM, Richard Hipp <d...@sqlite.org> wrote: > On Thu, Jan 19, 2012 at 7:51 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > Interesting: > > > > 1. There are no database schema changes occurring after the

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-19 Thread John Elrick
triggering the re-parse which did not do so in the earlier version. On Thu, Jan 19, 2012 at 7:45 AM, Richard Hipp <d...@sqlite.org> wrote: > On Thu, Jan 19, 2012 at 7:41 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > Question: > > > &g

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-19 Thread John Elrick
Question: If a query has already been prepared with sqlite3_prepare_v2, why would sqlite3_step need to call sqlite3Prepare, which in turn calls sqlite3RunParser? ___ sqlite-users mailing list sqlite-users@sqlite.org

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-18 Thread John Elrick
On Wed, Jan 18, 2012 at 4:32 PM, Richard Hipp <d...@sqlite.org> wrote: > On Wed, Jan 18, 2012 at 4:30 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > I made one more stab and narrowed down the mallocs to this call: > > > > Count:119,133

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-18 Thread John Elrick
started looking at yy_reduce, but it looks machine generated. If you give me some direction I'll see if I can narrow it down further. On Wed, Jan 18, 2012 at 4:02 PM, John Elrick <john.elr...@fenestra.com>wrote: > Whatever is happening, it appears I can only reproduce it under these > c

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-18 Thread John Elrick
Whatever is happening, it appears I can only reproduce it under these circumstances. I'll continue digging into the procedure tomorrow to see if I can narrow down where this is coming from. On Wed, Jan 18, 2012 at 3:54 PM, John Elrick <john.elr...@fenestra.com>wrote: > FYI, I traced t

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-18 Thread John Elrick
FYI, I traced the number of calls to sqlite3ResetInternalSchema() on a smaller set of data. It appears that it isn't the culprit. Total mallocs: 148,156 Total calls to sqlite3ResetInternalSchema(): 63 On Wed, Jan 18, 2012 at 3:41 PM, John Elrick <john.elr...@fenestra.com>wrote: >

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-18 Thread John Elrick
What else could trigger a call to sqlite3ResetInternalSchema()? I'm getting a clear breakpoint tracing back to such innocuous calls as _sqlite3_step. On Wed, Jan 18, 2012 at 3:31 PM, Richard Hipp <d...@sqlite.org> wrote: > On Wed, Jan 18, 2012 at 3:12 PM, John Elrick <john.elr...@

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-18 Thread John Elrick
I could not use any of the proffered tools for several reasons. After an inspiration yesterday, I dusted off my C programming skills and started hacking a pseudo-call stack system into SQLite3.c. I've put the statistics below for the top couple of call stacks. Total times _malloc called in

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-17 Thread John Elrick
On Mon, Jan 16, 2012 at 8:05 PM, John Elrick <john.elr...@fenestra.com>wrote: > > > On Mon, Jan 16, 2012 at 5:31 PM, Richard Hipp <d...@sqlite.org> wrote: > >> On Mon, Jan 16, 2012 at 5:13 PM, John Elrick <john.elr...@fenestra.com >> >wrote: >&

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-17 Thread John Elrick
On Mon, Jan 16, 2012 at 5:36 PM, Richard Hipp <d...@sqlite.org> wrote: > On Mon, Jan 16, 2012 at 5:31 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > SNIP > This is very curious and perhaps a useful clue. SQLite does call > SNIP It is a clue that I

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-17 Thread John Elrick
On Mon, Jan 16, 2012 at 11:47 PM, Max Vlasov <max.vla...@gmail.com> wrote: > On Tue, Jan 17, 2012 at 5:05 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > > I can try that approach with BCC. I was concerned that the IDE would be > > ge

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
On Mon, Jan 16, 2012 at 5:31 PM, Richard Hipp <d...@sqlite.org> wrote: > On Mon, Jan 16, 2012 at 5:13 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > I did this trace through the Delphi IDE and hand copied the call stack. > > Unfortunately, I can't

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
I will adjust the realloc tests tomorrow and let you know. The two routines are unique in the Delphi library unit. Thanks again for your time and efforts. On Mon, Jan 16, 2012 at 5:36 PM, Richard Hipp <d...@sqlite.org> wrote: > On Mon, Jan 16, 2012 at 5:31 PM, John Elrick

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
Richard, I don't know if the calls to _realloc will help also, but I've added tracing for those also. My updated output: Application: Surveyor.exe Sqlite version: 3.7.9 Date/Time:01/16/2012 17:20:58 Memory Used:

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
On Mon, Jan 16, 2012 at 5:14 PM, Simon Slavin <slav...@bigfraud.org> wrote: > > On 16 Jan 2012, at 9:37pm, John Elrick wrote: > > > Lacking the status interfaces for 3.6.17 I was forced to mark them as > > "interface not supported". Simon suggested I track th

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
I did this trace through the Delphi IDE and hand copied the call stack. Unfortunately, I can't give you line numbers, although I could go back and give you hex offsets from the start of the procedures. These are literally the first seven call stacks from _malloc after I initiate the trace. I'm

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
Addendum: This test was conducted using a full size test dataset. I had been using one that was an order of magnitude smaller for profiling before, but could use the full size for this test. On Mon, Jan 16, 2012 at 4:37 PM, John Elrick <john.elr...@fenestra.com>wrote: > Lacking t

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
Lacking the status interfaces for 3.6.17 I was forced to mark them as "interface not supported". Simon suggested I track the calls to _malloc, so I've added extensive tracing code. Here is the final output from the two versions: Application: Surveyor.exe Sqlite version:

Re: [sqlite] sql/tcl script

2012-01-16 Thread John Elrick
- stephan beal > http://wanderinghorse.net/home/stephan/ > http://gplus.to/sgbeal > ___ > sqlite-users mailing list > sqlite-users@sqlite.org > http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users > -- John Elrick Fenestra Technologies 540-868-1377 __

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
On Fri, Jan 13, 2012 at 5:58 PM, Richard Hipp <d...@sqlite.org> wrote: > On Fri, Jan 13, 2012 at 4:49 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > 3.6.17 > > > > Procedure % TimeTimeCalls > > _

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
On Mon, Jan 16, 2012 at 10:33 AM, Richard Hipp <d...@sqlite.org> wrote: > On Mon, Jan 16, 2012 at 10:29 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > > > > SQLite has lots of interfaces that can be used to determine performance > >

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
On Fri, Jan 13, 2012 at 5:58 PM, Richard Hipp <d...@sqlite.org> wrote: > On Fri, Jan 13, 2012 at 4:49 PM, John Elrick <john.elr...@fenestra.com > >wrote: > SNIP > SQLite has lots of interfaces that can be used to determine performance and > status informati

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-16 Thread John Elrick
On Mon, Jan 16, 2012 at 4:37 AM, Max Vlasov <max.vla...@gmail.com> wrote: > On Sun, Jan 15, 2012 at 3:35 AM, John Elrick <john.elr...@fenestra.com > >wrote: > > > On Fri, Jan 13, 2012 at 5:13 PM, Max Vlasov <max.vla...@gmail.com> > wrote: > > >

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread John Elrick
On Fri, Jan 13, 2012 at 5:13 PM, Max Vlasov wrote: > John, > I read your previous post and unfortunately, your conversation with Richard > didn't reveal much details. I worked with scenario similar tou yours > (Delphi + statically linked sqlite) for several years and during

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread John Elrick
On Sat, Jan 14, 2012 at 10:42 AM, Dan Kennedy <danielk1...@gmail.com> wrote: > On 01/14/2012 04:49 AM, John Elrick wrote: > >> >> So, obviously the problem is that _malloc is being called a much larger >> number of times by 3.7.9 in this particular circumstance -

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread John Elrick
On Fri, Jan 13, 2012 at 5:58 PM, Richard Hipp <d...@sqlite.org> wrote: > On Fri, Jan 13, 2012 at 4:49 PM, John Elrick <john.elr...@fenestra.com > >wrote: > > > > > 3.6.17 > > > > Procedure % TimeTimeCalls > > _

Re: [sqlite] Some pieces of the puzzle are coming together

2012-01-14 Thread John Elrick
On Fri, Jan 13, 2012 at 5:20 PM, Pavel Ivanov wrote: > > At this point I've exhausted my avenues of research. Does anyone have > any > > further ideas as to what may be going on which could cause this increase > in > > memory management activity? Or any suggestions of where

[sqlite] Some pieces of the puzzle are coming together

2012-01-13 Thread John Elrick
further ideas as to what may be going on which could cause this increase in memory management activity? Or any suggestions of where to look next? -- John Elrick Fenestra Technologies 540-868-1377 ___ sqlite-users mailing list sqlite-users@sqlite.org

[sqlite] Is there some hidden way to create a log record of Sqlite commands?

2012-01-11 Thread John Elrick
ear to be a built in mechanism for creating this type of log. Is this correct or am I missing something? -- John Elrick Fenestra Technologies 540-868-1377 ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listi

Re: [sqlite] Problems encountered while upgrading Sqlite from 3.6.7 to 3.7.9

2012-01-10 Thread John Elrick
On Tue, Jan 10, 2012 at 10:14 AM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 10, 2012 at 10:08 AM, John Elrick <john.elr...@fenestra.com>wrote: > >> On Tue, Jan 10, 2012 at 8:31 AM, Richard Hipp <d...@sqlite.org> wrote: >> > On Tue, Jan 10,

Re: [sqlite] Problems encountered while upgrading Sqlite from 3.6.7 to 3.7.9

2012-01-10 Thread John Elrick
On Tue, Jan 10, 2012 at 8:31 AM, Richard Hipp <d...@sqlite.org> wrote: > On Tue, Jan 10, 2012 at 8:14 AM, John Elrick <john.elr...@fenestra.com>wrote: > >> I will have to get back to you on this.  While running tests against >> isolated queries, I noticed an unusual

Re: [sqlite] Problems encountered while upgrading Sqlite from 3.6.7 to 3.7.9

2012-01-10 Thread John Elrick
3.6.7? > > On Mon, Jan 9, 2012 at 11:13 AM, Richard Hipp <d...@sqlite.org> wrote: > >> Please send private email to d...@sqlite.org.  CC to d...@sqlite.org and >> j...@sqlite.org. >> >> >> On Mon, Jan 9, 2012 at 11:09 AM, John Elrick <john.elr..

Re: [sqlite] Problems encountered while upgrading Sqlite from 3.6.7 to 3.7.9

2012-01-09 Thread John Elrick
Sorry "The combined files are 2.4 MB zipped" On Mon, Jan 9, 2012 at 11:09 AM, John Elrick <john.elr...@fenestra.com> wrote: > Richard, > > I have the information available for you.  Unfortunately, it appears > that I cannot actually reduce the case, however, the

Re: [sqlite] Problems encountered while upgrading Sqlite from 3.6.7 to 3.7.9

2012-01-09 Thread John Elrick
. The combined files are 2.4 MB. How would you prefer I send this file to you? On Fri, Jan 6, 2012 at 4:51 PM, John Elrick <john.elr...@fenestra.com> wrote: > Yes.  Look for it on Monday.  I'm about to leave for the day. The > queries are automatically generated and will require some min

Re: [sqlite] Problems encountered while upgrading Sqlite from 3.6.7 to 3.7.9

2012-01-06 Thread John Elrick
. On Fri, Jan 6, 2012 at 4:42 PM, Richard Hipp <d...@sqlite.org> wrote: > Can you publish for us: > > (1) A sample database > (2) The specific queries that are causing your problems > > So that we can investigate further? > > On Fri, Jan 6, 2012 at 4:31 PM, John Elrick &

[sqlite] Problems encountered while upgrading Sqlite from 3.6.7 to 3.7.9

2012-01-06 Thread John Elrick
be pleased to hear them. For the moment we are reverting to 3.6.7 as a temporary solution. -- John Elrick Fenestra Technologies 540-868-1377 ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] parallelizing an update

2010-02-02 Thread John Elrick
Robert Citek wrote: > Are there some white papers or examples of how to do updates in > parallel using sqlite? > > I have a large dataset in sqlite that I need to process outside of > sqlite and then update the sqlite database. The process looks > something like this: > > sqlite3 -separator $'\t'

Re: [sqlite] BUG: the rowid column in view is autom a tically named as id

2009-12-16 Thread John Elrick
Alexey Pechnikov wrote: > Hello! > > On Monday 14 December 2009 11:33:32 Jens Miltner wrote: > >> ... >> > > You did not understand me. The manually named columns > may not be renamed automatically. But in the test the > column "id" was renamed to "id:1": > > sqlite> create table test

Re: [sqlite] SQL question

2009-12-15 Thread John Elrick
Florian Schricker wrote: > Hello Simon, > SNIP > Please excuse me for being so blunt: > You have no idea what I'm supposed to do here nor do you have any idea > for whatever historic reason the database or table is designed "as is" > here nor (and finally) do you have no idea about my education

Re: [sqlite] is ticket 4037 not a bug?

2009-11-19 Thread John Elrick
hello sqlite> select myfoo.foo.* from myfoo.foo r; Error: near "*": syntax error sqlite> I cannot think of a good reason why: select myfoo.foo.* from myfoo.foo; would be expected to produce different results than: select r.* from myfoo.foo r; If the form

Re: [sqlite] LAN and exclusive lock

2009-10-21 Thread John Elrick
Jan wrote: > Hi, > > Although I read in a recent post by drh that it is not recommended to > use sqlite in a local network I would like to ask if the following > approach would work: > > A database in the local network needs to be accessed by about 20 people. > I suppose the max. number at the

Re: [sqlite] Question regarding BCC32

2009-10-13 Thread John Elrick
Cariotoglou Mike wrote: > Great. thanks for the info. however, this means that at least part of the > runtime library actually comes from the VCL, and not the BCC32 libraries, is > that not so ? > There are some OBJ files needed, we picked them up from somewhere, but I don't recall exactly

Re: [sqlite] Question regarding BCC32

2009-10-12 Thread John Elrick
Cariotoglou Mike wrote: > Has anybody had experience using Borland's free compiler (bcc32) to > build the sqlite dll ? We use bcc32 to create an .obj which is linked into our Delphi application. No issues here and we've been using it for three years on a project deployed to several hundred

Re: [sqlite] Improving select distinct performance

2009-10-08 Thread John Elrick
Jason Freshwater wrote: > If this comes out garbled again the answer is java 1.6 on 1.8Ghz windows xp > > i get about 500,000 rows/second reading single character values via xerial > jdbc > Thanks, I did some additional experiments with only integers and got similar results. I had always

Re: [sqlite] Improving select distinct performance

2009-10-08 Thread John Elrick
Jason Freshwater wrote: > 2. Read every record into my application and use a hashtable to > do my own select distinct. This is pretty good, about 0.6s to > achieve the distinct list (I can count each distinct value > along the way which is a useful side effect). Would you mind my asking what

Re: [sqlite] Synchronising multiple copies of a database

2009-10-08 Thread John Elrick
Jean-Denis Muys wrote: > On 10/7/09 21:35 , "Adam DeVita" wrote: > > >> One can also get a mess if Mr. Red and Mr Black both get new customers, and >> enter them and they both get the same ID because the auto-generated int >> happens to be the same. Both copies get

Re: [sqlite] Synchronising multiple copies of a database

2009-10-07 Thread John Elrick
Simon Slavin wrote: > On 7 Oct 2009, at 8:33pm, John Elrick wrote: > > >> Isn't this a variation of the DVCS problem? In other words, would it >> be correct in saying that the underlying issue is treating this as a >> database problem, rather than it being

Re: [sqlite] Synchronising multiple copies of a database

2009-10-07 Thread John Elrick
Simon Slavin wrote: > On 7 Oct 2009, at 7:20pm, Adam DeVita wrote: > > >> regarding this >> " The fault is that >> almost nobody does it right: they neglect to keep an 'unaltered >> central copy' and think they can cross-apply journals each time two >> databases talk to one-another. That does

  1   2   3   >