[sqlite] comma-separated string data

2014-04-04 Thread peter korinis
A data column in a link table contains comma-separated string data, where each value represents a value to link to another table. (many-to-many relationship) How do you 'parse' a table entry like: "4,66,51,3009,2,678, ." to extract these values and use them in an SQL statement, perhaps a WHERE

[sqlite] Performance HELP

2013-04-30 Thread peter korinis
This question regards SQLite (3.6) performance. (Lengthy because I want to describe the environment.) . Win-7 (64-bit, though I don't know if SQLite uses 64-bit). . 3 year old HP laptop with Intel Core Duo CPU P8400 @ 2.27GHz with only 4GB memory . 286GB HD (50% full) +

[sqlite] Performance HELP

2013-05-02 Thread peter korinis
Thank you for your responses. I was hoping for an easy hardware solution, like more memory or a faster HD . but it looks like indices and table design are where I need to focus. peterK ___ sqlite-users mailing list sqlite-users@sqlite.org

[sqlite] SQLite Use of Indexes

2013-07-08 Thread peter korinis
I have a large DB and want to improve query performance. Context: the DB contains 2 tables representing nationwide claim data: . a CLAIMS table = 43M rows with indices on claim_no and stateCounty code; and . a LINE table = 85M rows with indices on claim_no and HCPCS (a 5

Re: [sqlite] SQLite Use of Indexes

2013-07-09 Thread peter korinis
Thank you Simon for responding to my questions. Your phonebook (FName/LName) analogy clearly explained why 2 indices per table per select won't work. Let me provide a bit more info and a possible attempt to implement your suggestions for better indices. My 'bread and butter' query counts the

[sqlite] SQLite Use of Indices

2013-07-10 Thread peter korinis
t-Type: text/plain; charset=windows-1252 On 9 Jul 2013, at 6:06pm, peter korinis <kori...@earthlink.net> wrote: > So, to implement your suggestion of crafting better indices, here?s my approach: > 1. First action is joining the 2 tables on claim_no. {Therefore claim_no should

Re: [sqlite] SQLite Use of Indexes

2013-07-14 Thread peter korinis
From: "James K. Lowden" <jklow...@schemamania.org> To: sqlite-users@sqlite.org Subject: Re: [sqlite] SQLite Use of Indexes Message-ID: <20130712161038.b8b4df84.jklow...@schemamania.org> Content-Type: text/plain; charset=US-ASCII On Mon, 8 Jul 2013 15:32:21 -0400

[sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-01 Thread peter korinis
I'm new to SQLite . not a programmer . not a DBA . just an end-user with no dev support for a pilot project (single user, no updates, just queries). I want to analyze the data contained in a 44GB csv file with 44M rows x 600 columns (fields all <15 char). Seems like a DBMS will allow me to

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-02 Thread peter korinis
Solutions Operating Unit Northrop Grumman Information Systems From: sqlite-users-boun...@sqlite.org [sqlite-users-boun...@sqlite.org] on behalf of peter korinis [kori...@earthlink.net] Sent: Tuesday, May 01, 2012 3:06 PM To: sqlite-users@sqlite.org Subject: EXT

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-02 Thread peter korinis
thoroughly >tested for large data sources. > > > >Michael D. Black > >Senior Scientist > >Advanced Analytics Directorate > >Advanced GEOINT Solutions Operating Unit > >Northrop Grumman Information Systems > >________ &

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-03 Thread peter korinis
] On Behalf Of Warren Young Sent: Thursday, May 03, 2012 9:36 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] is SQLite the right tool to analyze a 44GB file On 5/1/2012 2:06 PM, peter korinis wrote: > Is SQLite the wrong tool for this project? Probably. SQLite is a data stor

Re: [sqlite] Data Import Techniques

2012-05-03 Thread peter korinis
Forgive me ... but what is SQLiteman? Will "import table data" help me load a csv file into a SQLite table? Thanks, peter -Original Message- From: sqlite-users-boun...@sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Simon Sent: Wednesday, May 02, 2012 3:03 PM To: General

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-04 Thread peter korinis
sqlite.org [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Oliver Peters Sent: Thursday, May 03, 2012 7:23 PM To: sqlite-users@sqlite.org Subject: Re: [sqlite] is SQLite the right tool to analyze a 44GB file Am 03.05.2012 19:59, schrieb peter korinis: > I have R but really haven't used it much. I know

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-04 Thread peter korinis
44GB file On Tue, May 01, 2012 at 04:06:01PM -0400, peter korinis wrote: > I'm new to SQLite . not a programmer . not a DBA . just an end-user > with no dev support for a pilot project (single user, no updates, just queries). > > > > I want to analyze the data contained in a 44GB

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-07 Thread peter korinis
Database Subject: Re: [sqlite] is SQLite the right tool to analyze a 44GB file On 4 May 2012, at 4:02pm, peter korinis <kori...@earthlink.net> wrote: > Sqlitespy looks good ... I will try it. > website says download contains sqlite itself, which I already have - > will there be a

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-07 Thread peter korinis
e or nothing more? I tried several different ways but still get same error when I try to import. Without knowing syntax, I'm forced to ask these dumb questions or give up (not) . since no good documentation - sorry. Thanks, Have you ran *.mode csv*? Jonas Malaco Filho 2012/5/7 peter ko

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-07 Thread peter korinis
On 7 May 2012, at 4:41pm, peter korinis wrote: > My input file is a comma-delimited text file > When I run .import I get the following "Error: FILE line 1: expected 46 > columns of data but found 1" > It seems .import is not recognizing comma delimiter.

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-07 Thread peter korinis
Thanks for advice http://www.pantz.org/software/sqlite/sqlite_commands_and_general_usage.html had the explanation/example I needed to get the import done successfully. Using ".separator ," was what I was missing. peter ___ sqlite-users

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-07 Thread peter korinis
I can't find a table I just created and imported data. With .databases I can see 'main' but with no location on disk and with . tables I see the table. Instructions says they are save . but where. I can't find them with windows search? pk ___

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-07 Thread peter korinis
Simon I searched the entire disk for the table name and no matches. pk ___ sqlite-users mailing list sqlite-users@sqlite.org http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] is SQLite the right tool to analyze a 44GB file

2012-05-07 Thread peter korinis
I did NOT specify a file name when I started sqlite3 - so I guess all is lost. I'll have to redo it. So then the first thing is to specify the db (file) name - what syntax? I tried typing a file name (test.db) but error. pk ___ sqlite-users