Re: [sqlite] Memoization in sqlite json1 functions

2017-03-25 Thread Deon Brewis
@mailinglists.sqlite.org> Subject: Re: [sqlite] Memoization in sqlite json1 functions > On Mar 24, 2017, at 4:48 PM, Deon Brewis <de...@outlook.com> wrote: > > Yeah ok, if you take I/O hits then things like memory pre-fetching makes zero > difference. We're more in the bu

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-25 Thread Jens Alfke
> On Mar 24, 2017, at 4:48 PM, Deon Brewis wrote: > > Yeah ok, if you take I/O hits then things like memory pre-fetching makes zero > difference. We're more in the business of "You take a page fault" == "You buy > more memory". Different level of performance requirements.

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-24 Thread Deon Brewis
ite-users-boun...@mailinglists.sqlite.org] On Behalf Of Jens Alfke Sent: Thursday, March 23, 2017 6:09 PM To: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> Subject: Re: [sqlite] Memoization in sqlite json1 functions > On Mar 23, 2017, at 3:17 PM, Deon Brewis <de...@outl

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-23 Thread Jens Alfke
> On Mar 23, 2017, at 3:17 PM, Deon Brewis wrote: > > If you however can use a forward-only push or pull parser like a SAX or StAX > parse, it's a different story. I'm using a StAX-like pull parser for a binary > json-ish internal format we have, and reading & parsing

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-23 Thread Deon Brewis
ster. - Deon -Original Message- From: sqlite-users [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Jens Alfke Sent: Thursday, March 23, 2017 11:05 AM To: SQLite mailing list <sqlite-users@mailinglists.sqlite.org> Subject: Re: [sqlite] Memoization in sqlite json1

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-23 Thread Domingo Alvarez Duarte
Hello Jens ! Nice to know this project, I'll look at it. Cheers ! On 23/03/17 15:05, Jens Alfke wrote: On Mar 23, 2017, at 4:30 AM, Richard Hipp wrote: BLOBs are reserved for a future enhancement in which BLOBs will store the binary encoding for JSON. I’ve been down this

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-23 Thread Jens Alfke
> On Mar 23, 2017, at 4:30 AM, Richard Hipp wrote: > > BLOBs are reserved for a future enhancement in which BLOBs will store the > binary encoding for JSON. I’ve been down this road. There are a number of JSON-compatible binary encoding formats, but most of them don’t save

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-23 Thread Scott Hess
What is the goal, though? Your app knows your data and performance needs, so if you find yourself running the same query to read off the same result set over and over, change your app to do the right thing. If it's somehow more convenient to have SQLite do it, populate a temporary table and pull

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-23 Thread Domingo Alvarez Duarte
Hello Richard ! Yes I do see "because the implementation parsing the JSON anew each time" and this is a recurring pattern in some sqlite functions, would be nice if we could have a "session/query/row" storage space to store query information that can be reused, for example on the json

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-23 Thread Richard Hipp
On 3/22/17, Domingo Alvarez Duarte wrote: > Hello Richard ! > > I noticed that sqlite do not use any memoization in json1 functions. > > For example jsonExtractFunc and others parse the json string every time > it's called even when the json string is the same. > > minimal

Re: [sqlite] Memoization in sqlite json1 functions

2017-03-23 Thread Deon Brewis
That would be nice. I've resorted to a few horrible hacks like this: SELECT parse(data), extract("name"), extract("address"), release(data) FROM some_table; It works, but it relies on LTR parsing of arguments (which it does now, but I seriously doubt is a guarantee), as well as