Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-07 Thread Graham Holden
Tuesday, August 07, 2018, 7:36:26 PM, Abramo Bagnarawrote: > I've verified that not only substr(data, start, len) loads entire row, > but with my surprise that also retrieving a sibling column (I've tried > to add another column "info blob not null") loads entire row (i.e. > including whole data

[sqlite] Save text file content in db: lines or whole file?

2018-08-07 Thread Abramo Bagnara
Il 07/08/2018 19:41, Abroży Nieprzełoży ha scritto: > substr(data, start, len) loads entire value and then substrs it. > sqlite3_blob_read reads the desired parts. I've verified that not only substr(data, start, len) loads entire row, but with my surprise that also retrieving a sibling column

Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-07 Thread Abroży Nieprzełoży
substr(data, start, len) loads entire value and then substrs it. sqlite3_blob_read reads the desired parts. 2018-08-06 20:59 GMT+02:00, Abramo Bagnara : > Il 04/08/2018 07:07, Abramo Bagnara ha scritto: >> Il 03/08/2018 23:53, Abroży Nieprzełoży ha scritto: >>> -- One table with whole files >>>

Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-06 Thread Abramo Bagnara
Il 04/08/2018 07:07, Abramo Bagnara ha scritto: > Il 03/08/2018 23:53, Abroży Nieprzełoży ha scritto: >> -- One table with whole files >> CREATE TABLE content( >> id INTEGER PRIMARY KEY, >> data BLOB NOT NULL >> ); >> -- And second table with line boundaries >> CREATE TABLE lines(

Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-03 Thread Abramo Bagnara
Il 03/08/2018 23:53, Abroży Nieprzełoży ha scritto: > -- One table with whole files > CREATE TABLE content( > id INTEGER PRIMARY KEY, > data BLOB NOT NULL > ); > -- And second table with line boundaries > CREATE TABLE lines( > id INTEGER NOT NULL REFERENCES content(id), >

Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-03 Thread Richard Hipp
On 8/3/18, Abroży Nieprzełoży wrote: > > I would also consider keeping the largest files external to the > database. https://www.sqlite.org/intern-v-extern-blob.html > See also https://www.sqlite.org/fasterthanfs.html Yes, it is a little faster to store larger files directly in the filesystem.

Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-03 Thread Abroży Nieprzełoży
-- One table with whole files CREATE TABLE content( id INTEGER PRIMARY KEY, data BLOB NOT NULL ); -- And second table with line boundaries CREATE TABLE lines( id INTEGER NOT NULL REFERENCES content(id), line_no INTEGER NOT NULL, bytes_from INTEGER NOT NULL,

[sqlite] Save text file content in db: lines or whole file?

2018-08-03 Thread Abramo Bagnara
Il 03/08/2018 21:50, Simon Slavin ha scritto: > On 3 Aug 2018, at 8:04pm, Abramo Bagnara wrote: > >> Some queries will need to extract the whole file, while other queries >> will need to extract the text for a range of lines. > > Can you give us an idea of how many lines you expect per text

Re: [sqlite] Save text file content in db: lines or whole file?

2018-08-03 Thread Simon Slavin
On 3 Aug 2018, at 8:04pm, Abramo Bagnara wrote: > Some queries will need to extract the whole file, while other queries > will need to extract the text for a range of lines. Can you give us an idea of how many lines you expect per text file ? Minimum and maximum for 90% of the files would be

[sqlite] Save text file content in db: lines or whole file?

2018-08-03 Thread Abramo Bagnara
I need to save text files (let say between 1 KB to 20 MB) in a SQLite DB. I see two possibilities: 1) save all the content in a single column: create table content(id integer not null primary key, text blob not null); 2) split the content in lines: create table