Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Keith Medcalf
On Wednesday, 10 April, 2019 08:12, Peng Yu wrote: >On 4/10/19, Keith Medcalf wrote: >> The first column is of strings ... >> Do you mean a single string as in "KerfufledAllaHasbalah" >> Or a "bunch of strings with some implied delimiter" such as >> "Kerfufled/Alla/Hasballah" where "/" is

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Hick Gunter
asking "how to emulate a non-relational implementation". -Ursprüngliche Nachricht- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Peng Yu Gesendet: Mittwoch, 10. April 2019 16:12 An: SQLite mailing list Betreff: Re: [sqlite] [EXTERNAL] compressed

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Graham Holden
Wednesday, April 10, 2019, 2:03:24 PM, Peng Yu wrote: > Given the much larger disk space required, for an occasional search of > the data, it seems that it makes no sense to use sqlite3 if disk space > is a major concern. Whether it "makes sense" to use SQLite or not, probably only you can

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Peng Yu
On 4/10/19, Keith Medcalf wrote: > > The first column is of strings ... > > Do you mean a single string as in "KerfufledAllaHasbalah" > Or a "bunch of strings with some implied delimiter" such as > "Kerfufled/Alla/Hasballah" where "/" is the separator between strings? > > If the latter, the data

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Keith Medcalf
traffic volume. >-Original Message- >From: sqlite-users [mailto:sqlite-users- >boun...@mailinglists.sqlite.org] On Behalf Of Peng Yu >Sent: Wednesday, 10 April, 2019 08:01 >To: SQLite mailing list >Subject: Re: [sqlite] [EXTERNAL] compressed sqlite3 database file? > >I

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Keith Medcalf
April, 2019 08:01 >To: SQLite mailing list >Subject: Re: [sqlite] [EXTERNAL] compressed sqlite3 database file? > >I don't know specifically what you refer to as data normalization. My >guess is something like this. But it is irrelevant to my case. > >https://www.studytonight.com/

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Peng Yu
I don't know specifically what you refer to as data normalization. My guess is something like this. But it is irrelevant to my case. https://www.studytonight.com/dbms/database-normalization.php For my specific TSV file, it has about 50 million rows and just two columns. The first column is of

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Hick Gunter
April 2019 15:03 An: SQLite mailing list Betreff: Re: [sqlite] [EXTERNAL] compressed sqlite3 database file? Here is the runtime of using "select where like" (with %) on a .sq3 file. real0m23.105s user0m12.765s sys 0m2.882s data from a Here is the runtime of zgrep (roughly equiva

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Peng Yu
Here is the runtime of using "select where like" (with %) on a .sq3 file. real0m23.105s user0m12.765s sys 0m2.882s Here is the runtime of zgrep (roughly equivalent, except that zgrep search for the whole line). real0m33.814s user0m40.927s sys 0m0.660s Given the much

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Peng Yu
> > Your comparison is unfair. A .gz file is COMPRESSED by gzip and not in any > > way human readable without uncompressing it first. But to store the file (and occasionally search the data), I would prefer 278MB instead of 1.5GB. With a .gz file, at least I can zcat and zgrep. > How big is the

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Simon Slavin
On 10 Apr 2019, at 7:33am, Hick Gunter wrote: > Your comparison is unfair. A .gz file is COMPRESSED by gzip and not in any > way human readable without uncompressing it first. How big is the result if > you compress the sqlite file through gzip? Or how big is the TSV file you get if you

Re: [sqlite] [EXTERNAL] compressed sqlite3 database file?

2019-04-10 Thread Hick Gunter
Your comparison is unfair. A .gz file is COMPRESSED by gzip and not in any way human readable without uncompressing it first. How big is the result if you compress the sqlite file through gzip? -Ursprüngliche Nachricht- Von: sqlite-users