Re: [sqlite] This is driving me nuts

2018-05-26 Thread x
Thanks for the input gents. I’m going to have to look into the memory stuff as it’s something I know little about and it’s near bedtime in Scotland. Warren, the actual programme does use the vector (it’s used to store the RowIDs). The console app mark 1 also did this (results were the same) bu

Re: [sqlite] This is driving me nuts

2018-05-26 Thread Warren Young
On May 26, 2018, at 9:43 AM, x wrote: > > I was sizing the vector beforehand and then timing how long it took to store > the RowIDs returned by an sqlite query. You’re not actually modifying the vector, only allocating it and then not using it. A sufficiently smart optimizer might optimize tha

Re: [sqlite] This is driving me nuts

2018-05-26 Thread Abroży Nieprzełoży
Collect some memory statistics before and after memory allocation and before and after query execution. #include #include #include void memoryStats() { PROCESS_MEMORY_COUNTERS_EX pmc; PERFORMANCE_INFORMATION pi; MEMORYSTATUSEX ms; Ze

Re: [sqlite] This is driving me nuts

2018-05-26 Thread Clemens Ladisch
x wrote: > const int Size[]={11200,10}; // 112 million, 1 billion > > v.resize(Size[i]); > > // NB the exact same code is executed whether i is 0 or 1. > The only thing that > // changes is the size() of v and v isn't even used in the >

Re: [sqlite] This is driving me nuts

2018-05-26 Thread x
I also added v.shrink_to_fit() after the resize and tried again with Size[0] and Size[1] swapped. The time difference reappeared with the second pass 4+ secs faster than the first. From: sqlite-users on behalf of Abroży Nieprzełoży Sent: Saturday, May 26, 20

Re: [sqlite] This is driving me nuts

2018-05-26 Thread x
Hi Abrozy. I inserted the line int64_t Mem=sqlite3_memory_highwater(0); immediately after the while loop and added Mem to the cout. The values returned were 2234704 (I==0) 2234768 (I==1). Tom From: sqlite-users on behalf of Abroży Nieprzełoży

Re: [sqlite] This is driving me nuts

2018-05-26 Thread Abroży Nieprzełoży
> I swapped Size[0] with Size[1] so that the resize was going from large to > small. The timings on each pass were then similar but were still around the > 24.xxx mark associated with the i==1 pass. Note the difference between size and capacity of the vector. Reducing the size does not necessarily

[sqlite] This is driving me nuts

2018-05-26 Thread x
I'm using c++ builder 10.2 tokyo on windows 10 and sqlite 3.23.1. I was working on code that stored RowIDs in a vector. I was sizing the vector beforehand and then timing how long it took to store the RowIDs returned by an sqlite query. By accident I sized the vector too big (by a factor of 10)