[sqlite] Restricting fast no-result query yields slow no-result query

2010-02-04 Thread Kelly Jones
I have a query that runs very quickly and returns no results: SELECT * FROM filebackup WHERE sha1='x'; However, the more restrictive query below runs very slowly, although it obviously can't have any results either: SELECT * FROM filebackup WHERE sha1='x' AND refid=0; I have indexes on both

[sqlite] Storing 3-byte ints efficiently

2009-11-04 Thread Kelly Jones
I imported a file with ~16.8M rows of 2 integers each (~33.6M ints total) into an SQLite db, no indexes. The ints are all < 16777216 (3 bytes) At 3 bytes/int, I thought the resulting db would be ~100M in size (plus some overhead), but it was actually 274M. How do I make sqlite3 store ints

[sqlite] Wiki consisting of rows in a db table

2009-10-09 Thread Kelly Jones
Consider a wiki that lets you edit rows in a db table. Each page is a row in the table, and has fields that anyone can edit. Like all wikis, it keeps a history of edits (including who made the edits), and lets you revert an edit, or even delete a row (page) completely. Has anyone implemented

Re: [sqlite] Multi-master replication through UPDATE and INSERT logging?

2009-09-17 Thread Kelly Jones
On 9/17/09, Simon Slavin <slav...@hearsay.demon.co.uk> wrote: > > On 17 Sep 2009, at 4:54pm, Kelly Jones wrote: > >> I want to do multi-master sqlite3 replication by editing sqlite3.c to >> log UPDATE/INSERT queries with timestamps, and then using another >

[sqlite] Multi-master replication through UPDATE and INSERT logging?

2009-09-17 Thread Kelly Jones
I want to do multi-master sqlite3 replication by editing sqlite3.c to log UPDATE/INSERT queries with timestamps, and then using another program to run those queries on the other masters. I looked at the sqlite3Insert() function in sqlite3.c, but couldn't find a variable that holds the query

[sqlite] Viable alternatives to SQL?

2009-08-27 Thread Kelly Jones
Many sites let you search databases of information, but the search queries are very limited. I'm creating a site that'll allow arbitrary SQL queries to my data (I realize I'll need to handle injection attacks). Are there other viable ways to query data? I read a little on "Business System 12"

[sqlite] SQLite3 replication

2009-07-18 Thread Kelly Jones
Is there any way to real-time replicate SQLite3 dbs across servers? I realize I could just rsync constantly, but this seems inefficient. I know SQLite3 uses a journal when making changes: could I use this journal for replication, similar to how MySQL uses bin-logging for replication? -- We're

[sqlite] SQLite3 immune from injection attacks if file is readonly?

2009-07-18 Thread Kelly Jones
On a website, I want to take a user's query "as is", save it to a userquery.txt, and then do: sqlite3 /path/to/mydb < userquery.txt where /path/to/mydb is a *read-only* file. Is there *any* risk of an injection attack here? Specifically, does sqlite3 have any shell escapes or any way to change

Re: [sqlite] SQLite3 data mode for emacs?

2009-06-21 Thread Kelly Jones
On 6/21/09, Kees Nuyt <k.n...@zonnet.nl> wrote: > On Sun, 21 Jun 2009 10:01:22 -0700, Kelly Jones > <kelly.terry.jo...@gmail.com> wrote: > >>Emacs' "forms mode" lets you edit a text file as though each line were >>a database record. >> >>

[sqlite] SQLite3 data mode for emacs?

2009-06-21 Thread Kelly Jones
Emacs' "forms mode" lets you edit a text file as though each line were a database record. Is there a similar mode that lets you edit data inside an sqlite3 db? -- We're just a Bunch Of Regular Guys, a collective group that's trying to understand and assimilate technology. We feel that

Re: [sqlite] Importing text file of numbers into INTEGER PRIMARY KEY aka rowid

2009-06-06 Thread Kelly Jones
On 6/6/09, P Kishor <punk.k...@gmail.com> wrote: > On Sat, Jun 6, 2009 at 8:28 PM, Kelly Jones<kelly.terry.jo...@gmail.com> > wrote: >> On 6/6/09, P Kishor <punk.k...@gmail.com> wrote: >>> On Sat, Jun 6, 2009 at 1:43 PM, Kelly Jones<kelly.terry.jo...@gma

Re: [sqlite] Importing text file of numbers into INTEGER PRIMARY KEY aka rowid

2009-06-06 Thread Kelly Jones
On 6/6/09, P Kishor <punk.k...@gmail.com> wrote: > On Sat, Jun 6, 2009 at 1:43 PM, Kelly Jones<kelly.terry.jo...@gmail.com> > wrote: >> I have a text file onenum.txt with just "1234\n" in it, and a db w/ >> this schema: >> >> sqlite>

[sqlite] Importing text file of numbers into INTEGER PRIMARY KEY aka rowid

2009-06-06 Thread Kelly Jones
I have a text file onenum.txt with just "1234\n" in it, and a db w/ this schema: sqlite> .schema CREATE TABLE test (foo INTEGER PRIMARY KEY); When I import, it fails as follows: sqlite> .import onenum.txt test Error: datatype mismatch Is sqlite3 treating "1234" as a string or something? Short

[sqlite] SQLite3 support for 64-bit unsigned integers

2009-05-25 Thread Kelly Jones
I tried inserting 2^63-1 and the two integers after it into an SQLite3 db, but this happened: SQLite version 3.6.11 Enter ".help" for instructions Enter SQL statements terminated with a ";" sqlite> CREATE TABLE test (test INT); sqlite> INSERT INTO test VALUES (9223372036854775807); sqlite> INSERT

[sqlite] Can SQLite3 handle OSM 150G data file?

2009-04-27 Thread Kelly Jones
I've seen many posts saying that SQLite2 can't handle OpenStreetMap's large planet.osm data file: http://planet.openstreetmap.org/planet-090421.osm.bz2 which is 5.4G bzip2 compressed, about 150G uncompressed. Can SQLite3 handle this? Has anyone tried? I tried to do this myself, but I'm on a

[sqlite] Aliased column can't use index outside of VIEWs?

2009-04-15 Thread Kelly Jones
I have a hideous query that looks like this: SELECT anf.name AS child, anf2.name||anf3.name||anf4.name AS parent [...] WHERE child='albuquerque' AND parent='newmexico'; which takes forever to run. However, when I replace 'child' with 'anf.name' in the WHERE clause, it runs lightning fast (as

Re: [sqlite] Existing dbs available by FTP/Torrent/etc?

2009-04-09 Thread Kelly Jones
On 4/9/09, P Kishor <punk.k...@gmail.com> wrote: > On Thu, Apr 9, 2009 at 6:23 PM, Kelly Jones <kelly.terry.jo...@gmail.com> > wrote: >> Given how cool SQLite is, has anyone created SQLite dbs of geonames, >> tycho2, or other large data sets that are available

[sqlite] Existing dbs available by FTP/Torrent/etc?

2009-04-09 Thread Kelly Jones
Given how cool SQLite is, has anyone created SQLite dbs of geonames, tycho2, or other large data sets that are available for download via FTP, HTTP, Torrent or similar mechanism? I realize I could dl the raw data, create tables, import the data, create indexes, etc, but it's much faster just to