Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Adrian Ho
On 15/7/19 11:25 PM, Nelson, Erik - 2 wrote: > Dominique Devienne wrote on Monday, July 15, 2019 2:41 AM > >> That's when you reach for virtual tables (and their "virtual indices"). >> I.e. you keep your data in native data-structures (Boost.MultiIndex in my >> case), >> and just provide a SQLite

Re: [sqlite] [EXTERNAL] Re: how to delete BLOB object from the data base

2019-07-15 Thread Keith Medcalf
>-Original Message- >From: Keith Medcalf [mailto:kmedc...@dessus.com] >Sent: Monday, 15 July, 2019 19:13 >To: 'SQLite mailing list' >Subject: RE: [sqlite] [EXTERNAL] Re: how to delete BLOB object from >the data base > > >Use one connection. Use one thread. As follows: > >sqlite *db;

Re: [sqlite] [EXTERNAL] Re: how to delete BLOB object from the data base

2019-07-15 Thread Keith Medcalf
Use one connection. Use one thread. As follows: sqlite *db; sqlite_stmt *beginstatement; sqlite_stmt *commitstatement; sqlite_stmt *insertstatement; sqlite_stmt *deletestatement; int i = 0; sqlite_open_v2('database.db', , ...); sqlite_prepare_v2(db, "begin immediate", , ...);

Re: [sqlite] how to delete BLOB object from the data base

2019-07-15 Thread Simon Slavin
On 15 Jul 2019, at 8:38pm, Zhu, Liang [AUTOSOL/ASSY/US] wrote: > I am getting the SQLite_locked error on delete, and my data from the select > statement are junk. I think you are saying that you get the result SQLITE_LOCKED. To do this you must be A) Using one database connection for two

Re: [sqlite] [EXTERNAL] Re: how to delete BLOB object from the data base

2019-07-15 Thread Zhu, Liang [AUTOSOL/ASSY/US]
We do the increment of 1000, when the record reaches number which dividable by 1000, we delete the record. -Original Message- From: sqlite-users On Behalf Of Robert Hairgrove Sent: Monday, July 15, 2019 4:00 PM To: sqlite-users@mailinglists.sqlite.org Subject: [EXTERNAL] Re: [sqlite]

Re: [sqlite] how to delete BLOB object from the data base

2019-07-15 Thread David Raymond
The questions coming to mind at the moment: What is the schema of the table holding the BLOBs? What is the normal size for the blobs? How are you doing the inserts and deletes? What journal mode are you using? I would think normal way to delete a record is the simple delete from blob_table

Re: [sqlite] how to delete BLOB object from the data base

2019-07-15 Thread Robert Hairgrove
On 15.07.19 21:38, Zhu, Liang [AUTOSOL/ASSY/US] wrote: ... I am inserting to the BLOB data into the table at every 250ms, I delete the oldest row at every 600ms, also I am reading the data from the database at every 10ms... How do you determine the "oldest" row? I believe the timestamps

[sqlite] how to delete BLOB object from the data base

2019-07-15 Thread Zhu, Liang [AUTOSOL/ASSY/US]
Sqlite Experts, I have a table contains the BLOB object, I am inserting to the BLOB data into the table at every 250ms, I delete the oldest row at every 600ms, also I am reading the data from the database at every 10ms. After almost of 100,000 insert, delete and select operations, I am

Re: [sqlite] SQLiteDataReader has already read all rows in first result set

2019-07-15 Thread Guy St-Denis
Issue is RESOLVED. After more experimentation, I realized that I was causing the Read() operations in the first resultset by doing this after the ExecuteReader() call: PS D:\temp> $dataReader FieldCount -- 2 2 2 It would seem that asking PowerShell to 'output'

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Jens Alfke
> On Jul 15, 2019, at 3:57 AM, Richard Hipp wrote: > > Yeah there is. SQLite has a high-level query language (SQL) that can > radically simplify application development. I guess it depends on your programming style. Most apps whose architecture I’m aware of* either wrap something like an

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread ardi
On Mon, Jul 15, 2019 at 12:57 PM Richard Hipp wrote: > > On 7/15/19, Jens Alfke wrote: > > > > If you’re going to keep your data in memory, there’s no good reason to use > > SQLite at all. > > Yeah there is. SQLite has a high-level query language (SQL) that can > radically simplify application

[sqlite] SQLiteDataReader has already read all rows in first result set

2019-07-15 Thread Guy St-Denis
New SQLite user... trying to understand the following issue. (Spent several hours on Google, Stack Overflow, SQLite mailing list archive... found nothing satisfactory.) [ISSUE] As far as I can tell, after calling ExecuteReader(), I cannot Read() any rows in the *first* result set because the

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Nelson, Erik - 2
Dominique Devienne wrote on Monday, July 15, 2019 2:41 AM >That's when you reach for virtual tables (and their "virtual indices"). >I.e. you keep your data in native data-structures (Boost.MultiIndex in my >case), >and just provide a SQLite view of it. Much faster than "pure-in-Memory" with >

Re: [sqlite] Bug report: crash when close blob handle after close_v2 db

2019-07-15 Thread Dan Kennedy
On 14/7/62 17:18, Chaoji Li wrote: This problem is only present for 3.28+. A sample test case is attached. Thanks for reporting this. We think it's fixed here:   https://sqlite.org/src/info/52f463d29407fad6 The mailing list stripped off your test case, so if you could either run it with

Re: [sqlite] "unable to use function highlight in the requested context" in group by

2019-07-15 Thread Jake Thaw
Dan Kennedy explained why this limitation exists: >On 27/2/62 05:47, Jake Thaw wrote: >>This may not strictly be a bug, but currently (3.27.2) a vtab cannot overload >>scalar functions in aggregate queries. >> >>Adding a check for TK_AGG_COLUMN in sqlite3VtabOverloadFunction makes my use >>case

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Richard Hipp
On 7/15/19, Jens Alfke wrote: > > If you’re going to keep your data in memory, there’s no good reason to use > SQLite at all. Yeah there is. SQLite has a high-level query language (SQL) that can radically simplify application development. -- D. Richard Hipp d...@sqlite.org

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Dan Kennedy
On 15/7/62 00:05, ardi wrote: Hi! I'm going to use sqlite as means of the file format I/O for applications. One of the critical points in file I/O is saving the file in a safe way, so that data loss cannot happen (or at least the risk of happening is minimized as much as possible).

Re: [sqlite] Link errors with SQLITE_OMIT_VIRTUALTABLE

2019-07-15 Thread Dan Kennedy
On 14/7/62 15:59, Orgad Shaneh wrote: Hi, In reply to https://www.mail-archive.com/sqlite-users@mailinglists.sqlite.org/msg113512.html. Can you please accept this patch? The patch doesn't seem all that intrusive, but is there a reason you can't build from canonical sources instead of

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Dominique Devienne
On Mon, Jul 15, 2019 at 8:24 AM Barry wrote: > For performance reasons I've had to write custom code where I have changing > data that I need to keep 'indices' on. I found it very difficult to get it > right, and very fragile. If I didn't need the performance, I would much > rather have used an

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Barry
If you use the SQLite backup api to 'backup' (save) the in-memory database to the disk database, an atomic transaction is used to write the data to the destination database. This means that all of SQLites usual guarantees apply: If the operation completes, the disk database will contain a

Re: [sqlite] Safe saving of in-memory db to disk file

2019-07-15 Thread Dominique Devienne
On Mon, Jul 15, 2019 at 7:26 AM Jens Alfke wrote: > > On Jul 14, 2019, at 10:05 AM, ardi wrote: > > Do you have any recommendation for saving the inmemory db in a safe way? > > If you’re going to keep your data in memory, there’s no good reason to use > SQLite at all. Just define custom model

Re: [sqlite] Bug when creating a table via select?

2019-07-15 Thread Dominique Devienne
On Mon, Jul 15, 2019 at 6:01 AM J. King wrote: > On July 14, 2019 11:56:15 p.m. EDT, Donald Shepherd < > donald.sheph...@gmail.com> wrote: > >sqlite> create table x(a int, b text, c real, d blob, e vartext, > fgarbage); > >sqlite> pragma table_info(x); > >0|a|int|0||0 > >1|b|text|0||0 >

Re: [sqlite] [EXTERNAL] BLOB and TEXT comparisons

2019-07-15 Thread Hick Gunter
https://sqlite.org/datatype3.html 4.1. Sort Order The results of a comparison depend on the storage classes of the operands, according to the following rules: •A value with storage class NULL is considered less than any other value (including another value with storage class NULL). •An