I have the distinct impression that you are attempting to convert a flat file into a naked table and pretending that the result is a (relational) database.
Please rethink your approach. There is a design process called "normalization" that needs to be done first. This will identify "entities" (with "attributes") and "relations" that will greatly reduce data duplication found in flat files. -----Ursprüngliche Nachricht----- Von: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] Im Auftrag von Peng Yu Gesendet: Mittwoch, 10. April 2019 15:03 An: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> Betreff: Re: [sqlite] [EXTERNAL] compressed sqlite3 database file? Here is the runtime of using "select where like" (with %) on a .sq3 file. real 0m23.105s user 0m12.765s sys 0m2.882s data from a Here is the runtime of zgrep (roughly equivalent, except that zgrep search for the whole line). real 0m33.814s user 0m40.927s sys 0m0.660s Given the much larger disk space required, for an occasional search of the data, it seems that it makes no sense to use sqlite3 if disk space is a major concern. On 4/10/19, Peng Yu <pengyu...@gmail.com> wrote: >> > Your comparison is unfair. A .gz file is COMPRESSED by gzip and not >> > in any way human readable without uncompressing it first. > > But to store the file (and occasionally search the data), I would > prefer 278MB instead of 1.5GB. With a .gz file, at least I can zcat > and zgrep. > >> How big is the result if you compress the sqlite file through gzip? > > 620MB > >> Or how big is the TSV file you get if you expand your .gz file ? > > 1.17GB > -- Regards, Peng _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users ___________________________________________ Gunter Hick | Software Engineer | Scientific Games International GmbH | Klitschgasse 2-4, A-1130 Vienna | FN 157284 a, HG Wien, DVR: 0430013 | (O) +43 1 80100 - 0 May be privileged. May be confidential. Please delete if not the addressee. _______________________________________________ sqlite-users mailing list sqlite-users@mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
