[sqlite] Suggestions for Fast Set Logic?

2015-07-20 Thread Andy Rahn
high energy > physics experiment results). > > -Urspr?ngliche Nachricht- > Von: Scott Robison [mailto:scott at casaderobison.com] > Gesendet: Montag, 13. Juli 2015 07:53 > An: sqlite-users at mailinglists.sqlite.org > Betreff: Re: [sqlite] Suggestions for Fast Set Logic?

[sqlite] Suggestions for Fast Set Logic?

2015-07-13 Thread Hick Gunter
iment results). -Urspr?ngliche Nachricht----- Von: Scott Robison [mailto:scott at casaderobison.com] Gesendet: Montag, 13. Juli 2015 07:53 An: sqlite-users at mailinglists.sqlite.org Betreff: Re: [sqlite] Suggestions for Fast Set Logic? On Sun, Jul 12, 2015 at 9:46 AM, James K. Lowden wrote: > On

[sqlite] Suggestions for Fast Set Logic?

2015-07-13 Thread Scott Robison
On Sun, Jul 12, 2015 at 9:46 AM, James K. Lowden wrote: > On Sat, 11 Jul 2015 19:02:59 -0600 > Scott Robison wrote: > > > > I don't follow you. A complex query is an assemblage of clauses. > > > Whether or not broken down "to individual sub queries", the search > > > arguments are the same.

[sqlite] Suggestions for Fast Set Logic?

2015-07-12 Thread James K. Lowden
On Sat, 11 Jul 2015 19:02:59 -0600 Scott Robison wrote: > > I don't follow you. A complex query is an assemblage of clauses. > > Whether or not broken down "to individual sub queries", the search > > arguments are the same. They are few in kind, even though they may > > be large in number.

[sqlite] Suggestions for Fast Set Logic?

2015-07-12 Thread Simon Slavin
On 12 Jul 2015, at 2:13am, Scott Robison wrote: > If you can know that, great. I'm just taking him at his word that arbitrary > complexity could result in table scans which are unacceptable. I don't think he has tried it. I think he has concluded that it will be too slow without having tried

[sqlite] Suggestions for Fast Set Logic?

2015-07-12 Thread Simon Slavin
On 12 Jul 2015, at 2:02am, Scott Robison wrote: > I'm just saying that knowing which index to create in advance is impossible > if the user can specify arbitrarily complex where clauses. You don't do that. You create a few indexes in advance based on the search combinations users are likely

[sqlite] Suggestions for Fast Set Logic?

2015-07-12 Thread Simon Slavin
On 12 Jul 2015, at 1:16am, James K. Lowden wrote: > Scott Robison wrote: > >> As described, the user will be able to create arbitrarily >> complex queries. Since it is impossible to know in advance what >> indexes might be required, breaking it down to individual sub queries >> with simple

[sqlite] Suggestions for Fast Set Logic?

2015-07-11 Thread James K. Lowden
On Fri, 10 Jul 2015 09:54:27 -0600 Scott Robison wrote: > As described, the user will be able to create arbitrarily > complex queries. Since it is impossible to know in advance what > indexes might be required, breaking it down to individual sub queries > with simple where clauses that can be

[sqlite] Suggestions for Fast Set Logic?

2015-07-11 Thread Scott Robison
On Jul 11, 2015 7:11 PM, "Simon Slavin" wrote: > > > On 12 Jul 2015, at 2:02am, Scott Robison wrote: > > > I'm just saying that knowing which index to create in advance is impossible > > if the user can specify arbitrarily complex where clauses. > > You don't do that. You create a few indexes

[sqlite] Suggestions for Fast Set Logic?

2015-07-11 Thread Scott Robison
On Jul 11, 2015 6:16 PM, "James K. Lowden" wrote: > > On Fri, 10 Jul 2015 09:54:27 -0600 > Scott Robison wrote: > > > As described, the user will be able to create arbitrarily > > complex queries. Since it is impossible to know in advance what > > indexes might be required, breaking it down to

[sqlite] Suggestions for Fast Set Logic?

2015-07-10 Thread Simon Slavin
On 10 Jul 2015, at 5:58pm, James K. Lowden wrote: > Simon Slavin wrote: > >> If you're going to do it properly you have a keywords column which >> contains strings like >> >> , >> ,animal, >> ,animal,predator, >> ,animal,predator,bird, > > Why do it that way? I would recommend a schema

[sqlite] Suggestions for Fast Set Logic?

2015-07-10 Thread Simon Slavin
On 10 Jul 2015, at 4:14pm, Andy Rahn wrote: > I don't think > there is a way I can just concatenate an extensive list of WHERE clauses > into one monolithic SQLite query and also guarantee that SQLite will be > able to SEARCH via indexes instead of falling back to slow SCANS. You are engaged

[sqlite] Suggestions for Fast Set Logic?

2015-07-10 Thread James K. Lowden
On Thu, 9 Jul 2015 22:28:04 +0100 Simon Slavin wrote: > If you're going to do it properly you have a keywords column which > contains strings like > > , > ,animal, > ,animal,predator, > ,animal,predator,bird, Why do it that way? I would recommend a schema like Keith's, with one keyword per

[sqlite] Suggestions for Fast Set Logic?

2015-07-10 Thread Andy Rahn
t; 'chocolate'); > > will return all the data tuples associated with the keyword ('once' or > 'bonca') and 'chocolate' > > > > -Original Message- > > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users- > > bounces at mailinglists.sqlite.org] On Be

[sqlite] Suggestions for Fast Set Logic?

2015-07-10 Thread Scott Robison
While I am opposed to premature optimization as well, this is not clearly a case of that. As described, the user will be able to create arbitrarily complex queries. Since it is impossible to know in advance what indexes might be required, breaking it down to individual sub queries with simple

[sqlite] Suggestions for Fast Set Logic?

2015-07-09 Thread Simon Slavin
On 9 Jul 2015, at 9:37pm, Igor Tandetnik wrote: > On 7/9/2015 11:25 AM, Andy Rahn wrote: > >> I want to build an application that can search for specific documents based >> on a lot of criteria from a user (e.g. matching keywords, text, etc...). >> And then to combine these results using

[sqlite] Suggestions for Fast Set Logic?

2015-07-09 Thread Keith Medcalf
sday, 9 July, 2015 11:26 > To: General > Subject: [sqlite] Suggestions for Fast Set Logic? > > I want to build an application that can search for specific documents > based > on a lot of criteria from a user (e.g. matching keywords, text, etc...). > And then to combine

[sqlite] Suggestions for Fast Set Logic?

2015-07-09 Thread Igor Tandetnik
On 7/9/2015 11:25 AM, Andy Rahn wrote: > I want to build an application that can search for specific documents based > on a lot of criteria from a user (e.g. matching keywords, text, etc...). > And then to combine these results using boolean logic ... For example, > keyword 'animal' AND rating > 3

[sqlite] Suggestions for Fast Set Logic?

2015-07-09 Thread Andy Rahn
I want to build an application that can search for specific documents based on a lot of criteria from a user (e.g. matching keywords, text, etc...). And then to combine these results using boolean logic ... For example, keyword 'animal' AND rating > 3 Each document has an integer id. My strategy