Re: [sqlite] kooky thought: a vm-only build (for embedded).feasible?

[dave]

...
> > size -- both code and RAM.  I know about the various 
> compile switches that
> > can turn off various features, but I wonder if I can really 
> strip it down
> > further by eliminating parsing, query planning, etc, 
> altogether, and only
> > support the virtual machine.  I do need virtual tables, 
> though.  In my
> > particular use-case, I only need read access -- no create 
> or update.  The
...
> > such that it will still work on memory-constrained devices 
> (e.g. having a
> > total of 128 KiB max for the whole system).
> >
> > Anway, has this been discussed before?  Or is it a fool's errand?
> 
> We did this once, back in 2005, for a startup company in Boston.  It
> was called "SSE".  Unfortunately, we didn't continue to support it.  I
> went looking for the source code and could not find it.
> 
> The database was to run on a smart-card with limited RAM.  All of the
> prepared statements were generated on a workstation, then serialized
> and stored in a special table in the database file.  The application
> would then use a special API that would deserialize a prepared
> statement (identified by a well-known integer) then bind parameters
> and run it.
> 
> So much has changed in the SQLite bytecode engine since then that
> there is basically zero chance that SSE would still run today, even if
> I could find the source code.
> 
> -- 
> D. Richard Hipp
> d...@sqlite.org

Ah, groovy.  Well, at least that is validation of the concept.  So it sounds
like I have a side project for my copious free time!

-dave


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] kooky thought: a vm-only build (for embedded). feasible?

[Richard Hipp]

On 4/15/18, dave  wrote:
> I had a stray thought, and wanted to ask if it's been thunk before,and if so
> what is the thinking?  Or just for commentary.
>
> I have been building a system, part of which uses sqlite and virtual tables.
> This is working great in a desktop/mobile environment.  However, eventually
> one day, I will want to migrate aspects of the product to deeply embedded
> systems (e.g. something like an STM32F4 class chip), and am thinking about
> size -- both code and RAM.  I know about the various compile switches that
> can turn off various features, but I wonder if I can really strip it down
> further by eliminating parsing, query planning, etc, altogether, and only
> support the virtual machine.  I do need virtual tables, though.  In my
> particular use-case, I only need read access -- no create or update.  The
> thinking being that I can build queries offline and compile them into the
> p-code (or whatever it's called), and either burn those well know queries
> into flash, or perhaps send them down the wire as needed.  Then of course
> (maybe even more critically), can I control ram usage in a deterministic way
> such that it will still work on memory-constrained devices (e.g. having a
> total of 128 KiB max for the whole system).
>
> Anway, has this been discussed before?  Or is it a fool's errand?

We did this once, back in 2005, for a startup company in Boston.  It
was called "SSE".  Unfortunately, we didn't continue to support it.  I
went looking for the source code and could not find it.

The database was to run on a smart-card with limited RAM.  All of the
prepared statements were generated on a workstation, then serialized
and stored in a special table in the database file.  The application
would then use a special API that would deserialize a prepared
statement (identified by a well-known integer) then bind parameters
and run it.

So much has changed in the SQLite bytecode engine since then that
there is basically zero chance that SSE would still run today, even if
I could find the source code.

-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] kooky thought: a vm-only build (for embedded).feasible?

[dave]

That was more-or-less my thinking.

Mostly, my inquiry is to solicit any advice or wisdom-of-the-ages, or even
advice against it.  After having sent that, I suspect that this would more
likely wind up being something I'm on my own in doing the hands-on work, but
I still welcome any advice on how to approach the surgery.

-dave

> -Original Message-
> From: sqlite-users 
> [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On 
> Behalf Of Simon Slavin
> Sent: Sunday, April 15, 2018 2:06 PM
> To: SQLite mailing list
> Subject: Re: [sqlite] kooky thought: a vm-only build (for 
> embedded).feasible?
> 
> 
> 
> 
> On 15 Apr 2018, at 7:54pm, dave <d...@ziggurat29.com> wrote:
> 
> > I wonder if I can really strip it down
> > further by eliminating parsing, query planning, etc, 
> altogether, and only
> > support the virtual machine.
> 
> I wonder what you would find if you looked through the data 
> structure of sqlite3_stmt.  Presumably the compilation 
> process would convert the SQL text into bytecode and the 
> bytecode would be stored in the statement.
> 
> Once you have seen the bytecode from your desired SQL, it 
> might be possible to write a C function which accepts a 
> pointer and a length and creates a statement with a copy of 
> that chunk of memory as the bytecode and everything else set 
> up the way it is in a newly-created statement.
> 
> Or something like that.
> 
> Simon.
> ___
> sqlite-users mailing list
> sqlite-users@mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
> 


___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] kooky thought: a vm-only build (for embedded). feasible?

[Simon Slavin]

On 15 Apr 2018, at 7:54pm, dave  wrote:

> I wonder if I can really strip it down
> further by eliminating parsing, query planning, etc, altogether, and only
> support the virtual machine.

I wonder what you would find if you looked through the data structure of 
sqlite3_stmt.  Presumably the compilation process would convert the SQL text 
into bytecode and the bytecode would be stored in the statement.

Once you have seen the bytecode from your desired SQL, it might be possible to 
write a C function which accepts a pointer and a length and creates a statement 
with a copy of that chunk of memory as the bytecode and everything else set up 
the way it is in a newly-created statement.

Or something like that.

Simon.
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] kooky thought: a vm-only build (for embedded). feasible?

[dave]

I had a stray thought, and wanted to ask if it's been thunk before,and if so
what is the thinking?  Or just for commentary.
 
I have been building a system, part of which uses sqlite and virtual tables.
This is working great in a desktop/mobile environment.  However, eventually
one day, I will want to migrate aspects of the product to deeply embedded
systems (e.g. something like an STM32F4 class chip), and am thinking about
size -- both code and RAM.  I know about the various compile switches that
can turn off various features, but I wonder if I can really strip it down
further by eliminating parsing, query planning, etc, altogether, and only
support the virtual machine.  I do need virtual tables, though.  In my
particular use-case, I only need read access -- no create or update.  The
thinking being that I can build queries offline and compile them into the
p-code (or whatever it's called), and either burn those well know queries
into flash, or perhaps send them down the wire as needed.  Then of course
(maybe even more critically), can I control ram usage in a deterministic way
such that it will still work on memory-constrained devices (e.g. having a
total of 128 KiB max for the whole system).
 
Anway, has this been discussed before?  Or is it a fool's errand?
 
Cheers!
 
-dave
___
sqlite-users mailing list
sqlite-users@mailinglists.sqlite.org
http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users