On 2018-07-10 21:17, Dan Kennedy wrote:
On 07/11/2018 02:56 AM, Nick wrote:
Using sqlite cli version 3.13 I have a simple schema with a virtual
FTS5 table providing full index searching. It is accessed by a python
application using apsw==3.13.0.post1.
I could successfully use the full index f
Simon,
No, we knew about normalisation, the database is normalised, that part
of the design hasn't changed in years.
The reasons for the massive reduction in database size is that we worked
out how to handle repeating lines of data that change just enough that
we thought they needed to be ke
Dan,
Thanks for the reply.
pragma secure_delete;
1
sqlite>
Which is a bit of a surprise as we have never seen it before. We had to
look it up using your reference.
No idea why that is set (if 1 means it is).
Should we simply ctrl-C the deletion, turn secure_delete off and then
try again? M
Simon,
No, we knew about normalisation, the database is normalised, that part
of the design hasn't changed in years.
The reasons for the massive reduction in database size is that we worked
out how to handle repeating lines of data that change just enough that
we thought they needed to be ke
On 17 Jul 2018, at 1:10pm, R Smith wrote:
> What kind of data did you store (maybe some examples if possible) that you
> could condense it by ~99% like that?
I think that the OP's organisation discovered the 'relational' part of RDBMS
and implemented normalisation.
To Rob Willett: Ryan Smith'
On 2018/07/17 1:20 PM, Rob Willett wrote:
Thanks for this. You make some interesting points about cache hits and
misses. Up until now, we hadn't seen a massive performance hit, whilst
some database operations didn't work as fast as we would like them to,
nothing works as fast we would like.
On 17 Jul 2018, at 12:20pm, Rob Willett wrote:
> Do you have any pointers to stuff we can read up on? We don't understand your
> comment "SQLite spent that whole time accessing your 50GB database file in an
> apparently random order." and would like to try and get more information
> about it.
Simon,
Thanks for this. You make some interesting points about cache hits and
misses. Up until now, we hadn't seen a massive performance hit, whilst
some database operations didn't work as fast as we would like them to,
nothing works as fast we would like.
I'm not familiar with these issues
On 17 Jul 2018, at 8:37am, Rob Willett wrote:
> I suspect that part of the issue is the VPS provider we use has a rate
> limiter on IOPS which is not normally an issue for us, but that might have
> slowed it down somewhat. However I don't think that it would have slowed it
> down by hours.
Ac
The top table finally finished around 07:00 this morning. It took 9.5
hours to drop the circa 190M rows.
I suspect that part of the issue is the VPS provider we use has a rate
limiter on IOPS which is not normally an issue for us, but that might
have slowed it down somewhat. However I don't th
10 matches
Mail list logo
