[sqlite] json1.c: isalnum(), isspace(), and isdigit() usage

2015-09-17 Thread Ralf Junker
On 17.09.2015 20:14, Scott Hess wrote: > The problem is that there are LOCALE settings where tolower() does things C > programmers don't expect. I think tr_TR was one case, the handling of 'I' > (Google "tr_tr locale bug" and you'll see lots of people hitting the same > general problem). It

[sqlite] Nuget Sqlite Packages

2015-09-17 Thread Simon Slavin
On 17 Sep 2015, at 8:27pm, R.Smith wrote: > On 2015-09-17 07:19 PM, Jim Boyce wrote: >> DISCLAIMER "This email and its attachments may contain privileged and/or >> confidential information//... > > It might also not contain anything at all. Actually, under the laws of both the European Union

[sqlite] Nuget Sqlite Packages

2015-09-17 Thread R.Smith
On 2015-09-17 07:19 PM, Jim Boyce wrote: > DISCLAIMER "This email and its attachments may contain privileged and/or > confidential information//... It might also not contain anything at all. :)

[sqlite] I don't understand how to use NOT EXISTS

2015-09-17 Thread R.Smith
On 2015-09-17 05:13 AM, Nicolas J?ger wrote: > @Ryan, >> To get you started on this road, may I ask you take a leap of faith >> and explain to us from A to Z what you would like to achieve with the >> tables and data you are asking about above - i.e don't show us the >> programming you think

[sqlite] I don't understand how to use NOT EXISTS

2015-09-17 Thread Keith Medcalf
> Some initial things. > Data needs to be kept in 1NF (or 1st-Normal-Form) which is relational > theory speak for "Everything without a 1-to-1 relationship with the key > field in a table, should be in another table". Many reasons for this > (if you care to read up on some RT) but the most

[sqlite] Using the json1 extension with Python

2015-09-17 Thread Charles Leifer
I spent some time this week figuring out how to compile ghaering's `pysqlite` library with support for the new SQLite json extension. Finally got it working and I must say it's pretty damn exciting! Here's a blog post I wrote that covers everything from compiling sqlite with JSON to running

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Rob Willett
Nicolas, Speed of development, ease of development, maintenance, available frameworks, available Perl modules. I actually know C the best of any language and will very happily code in C, but for some things its quicker to develop it in out in Perl. Certainly for anything involving

[sqlite] Handling the whole select query from an index

2015-09-17 Thread R.Smith
On 2015-09-17 04:50 PM, John McKown wrote: > On Thu, Sep 17, 2015 at 9:41 AM, R.Smith wrote: > >> (11) This forum. Need help or suggestions with how to optimize your >> internal value-store? Not so easy. Need help optimizing an SQLite DB or >> query? We've all seen the very clever people here

[sqlite] Handling the whole select query from an index

2015-09-17 Thread R.Smith
On 2015-09-17 02:22 PM, Richard Hipp wrote: > On 9/17/15, John McKown wrote: >> Well, this may be a heretical answer. > Yes, it is. > > There are a number of reasons to still use SQLite even if you are only > doing key/value lookups: > > (1) SQLite is the most widely deployed database engine in

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Rob Willett
John, The problem was the version of SQLite included in DBD. Its way old?. I am a sysadmin as well as developer as well as tester as well as CFO. I wear many hats in this startup :) Our conclusion is that we need to look at the architecture again to see if the web services can be speeded up.

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Rob Willett
John, The problem was the version of SQLite included in DBD. Its way old?. I am a sysadmin as well as developer as well as tester as well as CFO. I wear many hats in this startup :) Our conclusion is that we need to look at the architecture again to see if the web services can be speeded up.

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Rob Willett
Simon, We understand how SQlite is compiled into programs. We weren?t sure if the issues were Navicat which we sometimes use for a front end or the Perl DB stuff. My money was on Navicat being the problem. I was wrong, it was the DB version. The sqlite version is 3.7.17 on Ubuntu 14.04

[sqlite] Nuget Sqlite Packages

2015-09-17 Thread Scott Robison
On Thu, Sep 17, 2015 at 3:18 PM, Simon Slavin wrote: > > On 17 Sep 2015, at 8:27pm, R.Smith wrote: > > > On 2015-09-17 07:19 PM, Jim Boyce wrote: > >> DISCLAIMER "This email and its attachments may contain privileged > and/or confidential information//... > > > > It might also not contain

[sqlite] json1.c: isalnum(), isspace(), and isdigit() usage

2015-09-17 Thread Jan Nijtmans
2015-08-26 19:03 GMT+02:00 Ralf Junker : > ext/misc/json1.c uses the following functions from the C library: > > isalnum(): http://www.sqlite.org/src/artifact/541004e47235cefc?ln=564 > isspace(): http://www.sqlite.org/src/artifact/541004e47235cefc?ln=635 > isdigit():

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Simon Slavin
> On 17 Sep 2015, at 2:47pm, Rob Willett > wrote: > > 3. We cannot get WITHOUT ROWID working using Perl DBI. We get the following > error > > DBD::SQLite::db prepare failed: malformed database schema (postcode) - near > ?WITHOUT" > > This appears to be due to mismatched SQLite version but

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Rob Willett
John, Richard, A quick test shows the following: 1. Creating the database ?normally? without using WITHOUT ROWID. This has 1703538 rows. Doing 1,000 randomish accesses via the web service takes 10 secs. 2. Using a perl assoc array and doing the same access via a web service takes 10 secs.

[sqlite] user defined function returning a result set

2015-09-17 Thread Sylvain Pointeau
> > To do this, you'd have to create a virtual table that actually gets > instantiated per CSV file: > > CREATE VIRTUAL TABLE temp.file1 USING > csvFileReader('/path/to/my/file.csv'); > SELECT * FROM file1; > DROP TABLE file1; > > In the above, the xCreate method of the virtual

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Rob Willett
Simon, Fair point and agreed. We?ll update it. Rob > On 17 Sep 2015, at 13:56, Simon Slavin wrote: > > > On 17 Sep 2015, at 12:58pm, Rob Willett > wrote: > >> CREATE TABLE "postcode" ( >> "postcode" text NOT NULL, > > As an incidental tip not related to your question, I would

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Simon Slavin
On 17 Sep 2015, at 12:58pm, Rob Willett wrote: > CREATE TABLE "postcode" ( >"postcode" text NOT NULL, As an incidental tip not related to your question, I would recommend that you don't have a column name which is the same as a table name. Call the table 'locations' instead or

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Simon Slavin
On 17 Sep 2015, at 1:13pm, Richard Hipp wrote: > At the end of the CREATE TABLE statement add keywords: "WITHOUT > ROWID". Like this: > > CREATE TABLE postcode( > postcode TEXT, > long TEXT, > lat TEXT > ) WITHOUT ROWID; The documentation suggests that you need to explicitly declare a

[sqlite] json1.c: isalnum(), isspace(), and isdigit() usage

2015-09-17 Thread Scott Hess
On Thu, Sep 17, 2015 at 1:24 PM, Ralf Junker wrote: > On 17.09.2015 20:14, Scott Hess wrote: > >> The problem is that there are LOCALE settings where tolower() does things >> C >> programmers don't expect. I think tr_TR was one case, the handling of 'I' >> (Google "tr_tr locale bug" and you'll

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Rob Willett
John, Thanks for this. The base OS is Linux on a a 64bit server, the wrapper will be Perl running as a web service under Mojolicious. It will eventually be presented as a web service. We could simply load up the entire database into Perl into an Assoc array and search on that. Richard,

[sqlite] Nuget Sqlite Packages

2015-09-17 Thread Jim Boyce
AVISO LEGAL "As informa??es existentes nesta mensagem e nos arquivos anexados s?o para uso restrito. A utiliza??o, divulga??o, c?pia ou distribui??o dessa mensagem por qualquer pessoa diferente do destinat?rio ? proibida. Se essa mensagem foi recebida por engano, favor exclu?-la e informar ao

[sqlite] Suggestion: Regularize output of setting pragmas.

2015-09-17 Thread Scott Hess
Often, PRAGMA are documented like mmap_size, like: > Query or change the maximum number of bytes that are set aside > for memory-mapped I/O on a single database. The first > form (without an argument) queries the current limit. The > second form (with a numeric argument) sets the limit for the >

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Rob Willett
Hi, There was a topic on here a few weeks ago which I cannot remember and cannot find in the mess that is my e-mail system and after spending the last hours search the SQLite archives I still can?t find it so will ask here if anybody can remember or help. I can?t even format the question for

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Nicolas Jäger
hi Rob (Willet), May I ask why are you using Perl instead of C/C++ ? I'm not going to start a debate, I'm just curious. regards, Nicolas

[sqlite] json1.c: isalnum(), isspace(), and isdigit() usage

2015-09-17 Thread Scott Hess
The problem is that there are LOCALE settings where tolower() does things C programmers don't expect. I think tr_TR was one case, the handling of 'I' (Google "tr_tr locale bug" and you'll see lots of people hitting the same general problem). It isn't a problem of type safety, it's a problem that

[sqlite] Handling the whole select query from an index

2015-09-17 Thread John McKown
On Thu, Sep 17, 2015 at 9:41 AM, R.Smith wrote: > > > On 2015-09-17 02:22 PM, Richard Hipp wrote: > >> On 9/17/15, John McKown wrote: >> >>> Well, this may be a heretical answer. >>> >> Yes, it is. >> >> ?? ?? > (11) This forum. Need help or suggestions with how to optimize your > internal

[sqlite] Handling the whole select query from an index

2015-09-17 Thread John McKown
The latest PERL DBI for SQLite that I could see is at http://www.cpan.org/authors/id/I/IS/ISHIGAKI/DBD-SQLite-1.48.tar.gz . I took a look at it and it has sqlite3.c (and others) from the 3.8.10 amalgamation in it. Just "for fun", I copied 3 files (sqlite3.c, sqlite3.h, and sqlite3ext.h) from my

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Richard Hipp
On 9/17/15, John McKown wrote: > Well, this may be a heretical answer. Yes, it is. There are a number of reasons to still use SQLite even if you are only doing key/value lookups: (1) SQLite is the most widely deployed database engine in the world, so it is more likely to already be on the

[sqlite] Handling the whole select query from an index

2015-09-17 Thread John McKown
On Thu, Sep 17, 2015 at 7:21 AM, Rob Willett wrote: > John, > > Thanks for this. > > The base OS is Linux on a a 64bit server, the wrapper will be Perl running > as a web service under Mojolicious. It will eventually be presented as a > web service. > > We could simply load up the entire

[sqlite] Handling the whole select query from an index

2015-09-17 Thread Richard Hipp
On 9/17/15, Rob Willett wrote: > > The specific question I have is about trying to provide the fastest response > possible to a select query. I recall that the e-mail talked about using an > index to satisfy the query and therefore never having to go out to get the > rest of the data from the

[sqlite] Handling the whole select query from an index

2015-09-17 Thread John McKown
Well, this may be a heretical answer. Given what you have said, I wouldn't even try to use SQLite. Well, at least not directly. I would use a "hash table". SQLite's indexing, if I understand correctly, is a B-Tree. And that is the _only_ option. What would likely work better is a "hash index". You

[sqlite] I don't understand how to use NOT EXISTS

2015-09-17 Thread R.Smith
On 2015-09-16 08:27 PM, Nicolas J?ger wrote: > Hi guys, > > so there is the two ways I made: > > FIRST ONE: > == > > ** Create a table wich will store the result of EXISTS, > > CREATE TABLE _TAG_EXISTS_RESULT_( NAME, BOOL ); > > > ** The next query has to INSERT 'evil little sister' with

[sqlite] I don't understand how to use NOT EXISTS

2015-09-17 Thread Nicolas Jäger
Hi, @Simon, > >CASE ( SElECT BOOL FROM _TAG_EXISTS_RESULT_ ) WHEN 0 > > CASE takes a value. You can't put a SQLite command in there. > actually, it's working if you put the command between (). I guess it's like the `` on linux (eg), mplayer `ls *.mp3` but maybe,even if it's working, you