[sqlite] Mozilla wiki 'avoid SQLite'

[Stephen Chrzanowski]

I as well disagree that "the higher language, the better".  The bloat .NET
adds to the application size for both processing, memory and drive usage is
astronomical.  Its like going back to Visual Basic and including the
interpreter in your "compiled" application with your source code basically
copy/pasted right into the final EXE output (Encrypted/obfuscated/whatever).

I don't write in .NET, but I do agree for a developer, it CAN be a lot
easier with everything that language has to offer.  The libraries and
functions and procedures and mechanisms and whatever else included to make
our jobs easier are numerous.  But optimization goes out the window when
your 'compiler' does nothing more than translate your code at run-time
instead of converting it into byte-code the OS will understand and deal
with more directly. Everything has to eventually work its way down to
byte-code, but including the overhead of translating the language at
run-time doesn't help in the least.

On Sun, Jun 14, 2015 at 6:42 PM, Scott Doctor  wrote:

> On 6/14/2015 3:00 PM, Simon Slavin wrote:
>
>> The result is that that higher level the language you write in, the
>> better.
>>
>
> I disagree. The use of languages higher than C result in slow bloated
> code. A language that calls a language that calls a language. Simple
> programs become multi-megabyte resource hogs. I agree that C compilers are
> able to optimize assembler code to a level that hand-coded assembler
> probably could not achieve. The problem is that higher level languages are
> not compiling to assembler, but compiling to the language below it.
>
> 
> Scott Doctor
> scott at scottdoctor.com
> --
>
>
>
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Mozilla wiki 'avoid SQLite'

[Stephen Chrzanowski]

Bottom line point, the general "feeling" of that page is to dissuade people
from even CONSIDERING using SQLite for future enhancements (Considering the
age of that page).  I won't get on board with that.

What I meant by "common sense" is more towards the hardware the software is
going to be used on.  A desktop PC running Win7 with 32k or 64k cluster
sizes via NTFS on a platter drive of any RPM is going to operate a LOT
differently than an 4k to 8k cluster sized SD card formatted either at
FAT32 or EXT3 on a phone or tablet, when we're talking about the nuts and
bolts and welds that hold the machine, OS, and software together.
Different latencies between OS and storage medium (Forget the software
making the calls at this point) SHOULD drive the "optimization" development
towards "platform particular" configurations.  None of that has anything to
do with SQLite in itself.  And when SQLite does enter the picture for
optimization, a really basic check is to look at the OS the software is
running on to find out what kind of file system it might be writing to,
then create/regenerate the database paging system based on a "best guess"
cluster size, or have a pre-built matrix to indicate what 'normal' drive
configurations are and build the database against what that matrix says
works well.  That is only ONE aspect that can SERIOUSLY decrease IO waits.

Basically, my impression of most of that page, which the information
contained within is absolutely correct IMO, is that they're targeting
mostly the wrong foundation.  I know that it is a page of "Things to watch
for" but targeting SQLite itself.. well.. I feel that the author is looking
through the wrong looking glass.

On Sun, Jun 14, 2015 at 11:57 AM, Scott Robison 
wrote:

> On Jun 14, 2015 9:43 AM, "Stephen Chrzanowski" 
> wrote:
> >
> > Most of that looks to be more like common sense things rather than SQLite
> > specific, so why they're calling out SQLite, I've no idea.  Also, this
> doc
> > was last modified more than a year ago.  Stuff has changed both within FF
> > as well as SQlite since then.
>
> They call out SQLite because that is the engine they embed in their
> infrastructure. They aren't worried about people in their code base trying
> to use MySQL, PgSQL, SQL Server, or Oracle. They're worried about their
> developers using SQLite when some other format would be more appropriate
> for some fuzzy value of "appropriate".
>
> One shouldn't use a hammer on screws or a screwdriver on nails. Use the
> right tool for the job. Sometimes FILE* based functionality is the right
> tool. Sometimes not. It all comes down to the fuzzy concept of appropriate.
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Mozilla wiki 'avoid SQLite'

[Scott Hess]

On Sun, Jun 14, 2015 at 1:19 PM, Richard Hipp  wrote:
> On 6/14/15, Scott Hess  wrote:
>> SQLite essentially gives you a set of
>> b-trees with syntactic sugar over them,
>
> SQL (and I speak in general terms here, not just of SQLite) provides
> way more than syntactic sugar over b-trees.  The syntactic sugar over
> b-trees part is the *least* of the many advantages of SQL.  Other more
> important features include:

To be clear, what I mean by this is that if you have a set of
operations which you could do in your native language, and you decide
to instead push them into SQLite, and you are concerned about
performance and memory use, then it really behooves you to be able to
describe how to accomplish those operations using SQLite's basic data
structures.  Many developers build their SQL assumptions in an
environment with a memory-constrained app server calling out to a
database server running on the beefiest machine their ops people can
acquire.  The set of things-that-are-appropriate are going to be
different in an environment where "Map a multi-gigabyte index into
memory" is a valid approach.

In case it isn't clear from my post (I have a bit of a head cold going
on, so I probably shouldn't be writing posts on public mailing lists),
IMHO SQLite has done a really solid job covering the bases for the
area of operations it has staked out, and has done a good job of not
adding features inappropriate to those goals.  It's just that IMHO
developers often make dubious assumptions about what _else_ SQLite can
do.

-scott

[sqlite] DB Browser 3.7.0 for SQLite released

[Justin Clift]

Quick note.  DB Browser 3.7.0 for SQLite has been released: :)

  https://github.com/sqlitebrowser/sqlitebrowser/releases/tag/v3.7.0

Win + OSX binaries are there, as is the source tarball.  Ubuntu PPA
should be done in a day or two, and hopefully the FreeBSD port will
updated in the near future as well. :)

+ Justin

[sqlite] User-defined types -- in Andl

[Petite Abeille]

> On Jun 14, 2015, at 4:01 PM, david at andl.org wrote:
> 
> First, I added a RECURSE() function to Andl, similar to the CTE in SQLite.

Nice.

> The Mandelbrot algorithm looks like this.

Could we see something more, hmmm, pedestrian? Perhaps a simple recursive 
query, showing, say, all the managers of an employee given the following 
structure: create table employee( id integer not null, manager_id integer, 
constraint  employee_pk primary key( id ), constraint employee_manager_fk 
foreign key( manager_id ) references employee( id )  )

[sqlite] SQLite 3.6.14.2 and malformed database corruption via multiple processes

[Simon Slavin]

On 14 Jun 2015, at 5:39pm, Mel Llaguno  wrote:

> We have an application with embedded SQLite 3.6.14.2 which is being accessed 
> by multiple processes. These processes connect to the database with a BEGIN 
> IMMEDIATE TRANSACTION. Under what circumstances, if any, can database 
> corruption actually occur (ruling out filesystem instability and power 
> loss/kernel panics)?

Can I ask you to read this page



and then post again with your guesses as to which section(s) might be relevant ?

As general advice, check the result codes from all SQLite calls, even if 
there's nothing you can do when they go wrong (e.g. sqlite_close()).  Often the 
first indication that you're overwriting SQLite memory is something weird like 
_finalize() returning something other than SQLITE_OK.

Simon.

[sqlite] Mozilla wiki 'avoid SQLite'

[Simon Slavin]

On 14 Jun 2015, at 4:43pm, Stephen Chrzanowski  wrote:

> Most of that looks to be more like common sense things rather than SQLite
> specific, so why they're calling out SQLite, I've no idea.

Because the SQLite engine is part of FireFox, and SQLite calls are very easy to 
do from parts of, and extensions to, FireFox.  This isn't true for any other 
database engine.  When you have a DBMS every problem looks like a schema.

Simon.

[sqlite] Is recursive CTE fully capable?

[Simon Slavin]

On 14 Jun 2015, at 4:09pm, James K. Lowden  wrote:

> Simon Slavin  wrote:
> 
>> There are plenty of queries which can be expressed in a SQL database
>> but can't be answered without a computer which can reprogram itself
> 
> Are there?  Do you mean there are SQL queries like that?  Or do you
> mean there are such queries that could be asked of a relational database
> but cannot be expressed in SQL?

I was a little loose in my language.  There are queries which can be expressed 
in terms of tables of rows of columns, with numbers and text in them.  E.g

"Jane is four years old, Derek was twice Jane's age last time there was a 
Winter Olympics.  If Derek's dog is white with black spots how much does Jane's 
mother earn per year ?"

There are some queries where you can figure out a schema to store data to 
express the situation, but you can't figure out any combination of database 
massaging commands (INSERT/UPDATE/DELETE) and a final SELECT that will fetch 
the answer you want.

> There are queries that cannot be formulated in first order predicate
> logic, and recursion is the single capability of SQL that exceeds
> FOPL power.  Unless the recursion is circular, I don't see how an SQL
> query over a finite database could fail to terminate.

The Termination Problem is not about whether a SQL query terminates, but 
whether a program in another language terminates.  Suppose you have a TABLE of 
lines of a program in your favourite toy programming language: row 1 of the 
table is line 1 of the program, row 2 of the table is line 2 of the program, 
etc..  The programming language can be any reasonably complicated one.  BASIC, 
PASCAL, Java, JavaScript, whatever.

The task is to write some SQL code, including as many INSERT/UPDATE/DELETEs as 
you want to make other tables with information about the program, with a final 
SELECT which returns TRUE if and only if the program will halt.

Simon.

[sqlite] SQLite 3.6.14.2 and malformed database corruption via multiple processes

[Mel Llaguno]

We have an application with embedded SQLite 3.6.14.2 which is being accessed by 
multiple processes. These processes connect to the database with a BEGIN 
IMMEDIATE TRANSACTION. Under what circumstances, if any, can database 
corruption actually occur (ruling out filesystem instability and power 
loss/kernel panics)?


Here is an example of the DB state when corruption has already occurred and we 
run check-integrity :


Invalid row detected in redacted column row 3093: No row 3091 in table 
TableNameRedacted
Invalid row detected in redacted column row 3100: No row 3100 in table 
TableNameRedacted
File: '/path/to/sqlite/database' database disk image is malformed

Thanks,


Mel Llaguno

[sqlite] Mozilla wiki 'avoid SQLite'

[Simon Slavin]

On 14 Jun 2015, at 2:08pm, Jean Chevalier  wrote:

> Somewhat contradictory the Mozilla Foundation being a member of the SQLite 
> Consortium while their performance wiki prominently features a warning to 
> developers against using SQLite allegedly for performance reasons. Guard me 
> from my friends...
> 
> http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature

It's an interesting page, and I have no argument with most of it.  What they're 
doing is not comparing SQLite and some other database engine, but comparing 
SQLite with a flat text file.  It can be temping to write less code and use 
SQLite as if it's the perfect solution to every storage problem, but sometimes 
if you have something simple to store and search, nothing beats a plaintext 
file.  You just have to write your search code yourself.

That page is a good summary of many 'gotchas' for using those using SQLite for 
the first time.  As long as they don't take it too seriously.

Simon.

[sqlite] Mozilla wiki 'avoid SQLite'

[Darko Volaric]

The irony of your comment (which I entirely agree with) is that because
SQLite (and similar) does so much incredibly important stuff for you, it
ends up being difficult for programmers to use it, especially the lower
level the programming is. They don't understand the model of how the system
works, and how its meant to be used. Quite often they think they can put a
value in a field, and all will be well, because that's what SQL promises.
They don't think about transactions, normalization, referential integrity
or anything else, because they don't need those features. But unfortunately
if you don't then SQL will perform poorly (or incorrectly) because it isn't
a simple system to store data.

For example, I remember encountering a system where the programmers tracked
a trade though various states. Each time it changed states the system would
move (ie, insert then delete) the record from one table, into a nearly
identical table (without a transaction). They essentially used the table to
denote the state. Might make sense in an imperative model, where buffers
are common, but not at all in SQL.

All they're saying here is stop misusing SQLite: if a child cuts itself
with a knife, take away the knife.

On Sun, Jun 14, 2015 at 1:19 PM, Richard Hipp  wrote:

> On 6/14/15, Scott Hess  wrote:
> > SQLite essentially gives you a set of
> > b-trees with syntactic sugar over them,
>
> SQL (and I speak in general terms here, not just of SQLite) provides
> way more than syntactic sugar over b-trees.  The syntactic sugar over
> b-trees part is the *least* of the many advantages of SQL.  Other more
> important features include:
>
> (1) Transactions.  Atomic updates with the option to ROLLBACK at any time.
>
> (2) Data abstraction.  The schema concisely and accurately describes
> the data format in a way that is portable across systems and across
> time.  Content stored in an SQL database is far, far more likely to be
> accessible 25 years in the future.  It is also far more likely to be
> accessible today by other applications written in different
> programming languages or implemented on obscure and/or experimental
> hardware or operating systems.
>
> (3) Declarative Programming.  With SQL, the programmer asks the
> machine a question and lets the query planner figure out an
> appropriate algorithm.  A few lines of query text replace hundreds or
> thousands of lines of procedural code needed to implement that query.
> If performance problems are encountered, they can usually be remedied
> by CREATE INDEX, and without changing a single line of code - the
> query procedures devised by the query planner shift automatically.
>
> To be sure, many programmers end up using SQL as a glorified key-value
> store.  You can actually find programs that do "SELECT * FROM table
> WHERE pk=?" on two or three tables and then implement a join as
> procedural code in the application.  Maybe this is what they were
> taught.  Or maybe they never really understood the concept of a join.
> I dunno.  But for whatever reason, you do see a lot of people treating
> SQL as little more than an interface or wire protocol for talking to a
> btree, and thereby bypassing 95% of the goodness of SQL.   But just
> because a lot of programmers do this, does not mean it is right.
>
> Alexander Lloyd (on the Google Spanner team) says that "NoSQL
> databases ... enforce a broadly applied premature optimization on the
> entire system."  What Alexander means is that there are some
> specialized cases where NoSQL-type solutions are required for
> performance, but that those situations are the exception rather than
> the rule.   It is better to use the powerful abstractions provided by
> SQL to build a reliable system, then go back and optimize any
> performance bottlenecks on a case-by-case basis.  Alexander's remarks
> implicitly but clearly reference Don Knuth's assertion that premature
> optimization is the root of all evil.
>
> Another way of thinking about the difference between SQL and NoSQL is
> by analogy to C versus assembly language.  There was a huge debate in
> the software industry in the 1980s about whether applications should
> be coded in assembly or C.  Hand-coded assembly is theoretically
> faster than machine-code generated by a C compiler (or at least it was
> in the 80s - that point is debatable now).  But in practice, programs
> written in C tended to perform better than those written in assembly.
> The reason is that C being higher level enabled programmers to spend
> less time fiddling with bits and more time developing better
> algorithms.  C enables a programmer to spend more time "heads-up"
> thinking about solving the users problem and less them "heads-down"
> worrying about the details of the implementation.  The SQL vs. NoSQL
> situation is quite similar.  You can, in theory, always run faster
> with NoSQL storage.  But in practice, SQL is so much more powerful
> that applications coded using SQL tend to perform better, be more
> 

[sqlite] Mozilla wiki 'avoid SQLite'

[Richard Hipp]

On 6/14/15, Scott Hess  wrote:
> SQLite essentially gives you a set of
> b-trees with syntactic sugar over them,

SQL (and I speak in general terms here, not just of SQLite) provides
way more than syntactic sugar over b-trees.  The syntactic sugar over
b-trees part is the *least* of the many advantages of SQL.  Other more
important features include:

(1) Transactions.  Atomic updates with the option to ROLLBACK at any time.

(2) Data abstraction.  The schema concisely and accurately describes
the data format in a way that is portable across systems and across
time.  Content stored in an SQL database is far, far more likely to be
accessible 25 years in the future.  It is also far more likely to be
accessible today by other applications written in different
programming languages or implemented on obscure and/or experimental
hardware or operating systems.

(3) Declarative Programming.  With SQL, the programmer asks the
machine a question and lets the query planner figure out an
appropriate algorithm.  A few lines of query text replace hundreds or
thousands of lines of procedural code needed to implement that query.
If performance problems are encountered, they can usually be remedied
by CREATE INDEX, and without changing a single line of code - the
query procedures devised by the query planner shift automatically.

To be sure, many programmers end up using SQL as a glorified key-value
store.  You can actually find programs that do "SELECT * FROM table
WHERE pk=?" on two or three tables and then implement a join as
procedural code in the application.  Maybe this is what they were
taught.  Or maybe they never really understood the concept of a join.
I dunno.  But for whatever reason, you do see a lot of people treating
SQL as little more than an interface or wire protocol for talking to a
btree, and thereby bypassing 95% of the goodness of SQL.   But just
because a lot of programmers do this, does not mean it is right.

Alexander Lloyd (on the Google Spanner team) says that "NoSQL
databases ... enforce a broadly applied premature optimization on the
entire system."  What Alexander means is that there are some
specialized cases where NoSQL-type solutions are required for
performance, but that those situations are the exception rather than
the rule.   It is better to use the powerful abstractions provided by
SQL to build a reliable system, then go back and optimize any
performance bottlenecks on a case-by-case basis.  Alexander's remarks
implicitly but clearly reference Don Knuth's assertion that premature
optimization is the root of all evil.

Another way of thinking about the difference between SQL and NoSQL is
by analogy to C versus assembly language.  There was a huge debate in
the software industry in the 1980s about whether applications should
be coded in assembly or C.  Hand-coded assembly is theoretically
faster than machine-code generated by a C compiler (or at least it was
in the 80s - that point is debatable now).  But in practice, programs
written in C tended to perform better than those written in assembly.
The reason is that C being higher level enabled programmers to spend
less time fiddling with bits and more time developing better
algorithms.  C enables a programmer to spend more time "heads-up"
thinking about solving the users problem and less them "heads-down"
worrying about the details of the implementation.  The SQL vs. NoSQL
situation is quite similar.  You can, in theory, always run faster
with NoSQL storage.  But in practice, SQL is so much more powerful
that applications coded using SQL tend to perform better, be more
reliable, and have more and useful features than those written using
NoSQL or ad hoc storage solutions.

One clear example of an application suffering from a lack of SQL is
Git.  Git uses a NoSQL ad hoc key/value data store - files in the .git
directory.  This is fast (they say).  But it also causes serious
problems.  For example, finding the children of a commit involves a
full-table scan of the Git log.  This turns out to be so difficult and
slow that nobody implements it.  On GitHub (and on every other Git
interface I've seen) you will find links to the parents of a check-in
but not its children.  And so when looking at the history of a Git
project, it is impossible to find out what happened next.  It also
leads to the vexing problem known as "detached head state".  If the
Git log had been implemented as an SQL database (SQLite would have
been a good choice for this, I think, but there are other options)
then a detached head becomes impossible - that scourge of so many
hapless Git users simple would have never existed.  And finding (for
example) the first 30 descendants of a check-in would become a simple
query something like this:

WITH RECURSIVE
  dx(h,m) AS (SELECT child, mtime FROM gitlog WHERE child=$commithash
  UNION
  SELECT child, mtime FROM gitlog, dx WHERE parent=h
  ORDER BY 2)
SELECT h, 

[sqlite] Mozilla wiki 'avoid SQLite'

[Scott Doctor]

On 6/14/2015 3:00 PM, Simon Slavin wrote:
> The result is that that higher level the language you write in, the better.

I disagree. The use of languages higher than C result in slow 
bloated code. A language that calls a language that calls a 
language. Simple programs become multi-megabyte resource hogs. I 
agree that C compilers are able to optimize assembler code to a 
level that hand-coded assembler probably could not achieve. The 
problem is that higher level languages are not compiling to 
assembler, but compiling to the language below it.


Scott Doctor
scott at scottdoctor.com
--

[sqlite] Mozilla wiki 'avoid SQLite'

[Jean Chevalier]

Somewhat contradictory the Mozilla Foundation being a member of the SQLite 
Consortium while their performance wiki prominently features a warning to 
developers against using SQLite allegedly for performance reasons. Guard me 
from my friends...

http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature

[sqlite] Mozilla wiki 'avoid SQLite'

[Rich Shepard]

On Sun, 14 Jun 2015, Richard Hipp wrote:

> SQL (and I speak in general terms here, not just of SQLite) provides way
> more than syntactic sugar over b-trees. The syntactic sugar over b-trees
> part is the *least* of the many advantages of SQL. Other more important
> features include:

   Very well written, Richard.

> (3) Declarative Programming.  With SQL, the programmer asks the machine a
> question and lets the query planner figure out an appropriate algorithm. A
> few lines of query text replace hundreds or thousands of lines of
> procedural code needed to implement that query. If performance problems
> are encountered, they can usually be remedied by CREATE INDEX, and without
> changing a single line of code - the query procedures devised by the query
> planner shift automatically.

   Another stumbling block for some application developers is not recognizing
that SQL works in sets unlike earlier database formats that stored fields in
records. That's why a SELECT returns an entire set of table rows meeting the
selection criteria and the row order is not guaranteed.

Rich

[sqlite] Mozilla wiki 'avoid SQLite'

[Stephen Chrzanowski]

Most of that looks to be more like common sense things rather than SQLite
specific, so why they're calling out SQLite, I've no idea.  Also, this doc
was last modified more than a year ago.  Stuff has changed both within FF
as well as SQlite since then.

I've just deleted the blog I was going to write here, since I know I'm
pissing people off with my long thoughts (And I'm tired), so yeah...  I'll
cut it to a paragraph.

*Old data, common sense to clean up sometime during the current apps life
cycle, or something completely different spawned after the primary apps
life cycle with low priority.  *Proper knowledge of how SQLite handles
indexing of data is required (Looked at that bug report linked).
*Fragmentation happens, within and outside databases.  *Database page sizes
should be close to your "common" target platforms disk cluster size to
reduce disk IO.  *And yeah... I LOL'd at "JSON files or log files will show
better I/O patterns almost every time, especially if they're compressed and
read/written in entirety each time", specifically at JSON, and the concept
that substituting disk IO for CPU time to decompress then ANALYZE the full
content of that data, then delete previous data to recompress data is any
better, ESPECIALLY when trickling data into a compressed JSON log file.
Not a fan of that JSON fanboi tech at all.

On Sun, Jun 14, 2015 at 9:08 AM, Jean Chevalier  wrote:

> Somewhat contradictory the Mozilla Foundation being a member of the SQLite
> Consortium while their performance wiki prominently features a warning to
> developers against using SQLite allegedly for performance reasons. Guard me
> from my friends...
>
>
> http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
>

[sqlite] Is recursive CTE fully capable?

[James K. Lowden]

On Fri, 12 Jun 2015 01:45:50 +0100
Simon Slavin  wrote:

> There are plenty of queries which can be expressed in a SQL database
> but can't be answered without a computer which can reprogram itself

Are there?  Do you mean there are SQL queries like that?  Or do you
mean there are such queries that could be asked of a relational database
but cannot be expressed in SQL?  

There are queries that cannot be formulated in first order predicate
logic, and recursion is the single capability of SQL that exceeds
FOPL power.  Unless the recursion is circular, I don't see how an SQL
query over a finite database could fail to terminate.  

--jkl

[sqlite] Mozilla wiki 'avoid SQLite'

[J.B. Nicholson-Owens]

Jean Chevalier wrote:
> Somewhat contradictory the Mozilla Foundation being a member of the
> SQLite Consortium while their performance wiki prominently features a
> warning to developers against using SQLite allegedly for performance
> reasons. Guard me from my friends...
>
> http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature

That page describes situations where SQLite doesn't give best available 
performance, reduce complexity, or provide needed features, and then 
goes on to justify those conclusions (including recognizing how SQLite 
isn't unique in these respects: "This isn't an indictment of SQLite 
itself -- any other relational embedded DB would pose the same 
challenges."). I'm not sure what point you were raising in your post, so 
I'll have to guess. Your summary suggests that you expected SQLite 
Consortium members will endorse SQLite even in situations where SQLite 
isn't a good fit due to preferring other tradeoffs.

Are SQLite Consortium members somehow obliged to endorse SQLite for 
tasks even when other approaches present more desirable tradeoffs? If 
so, where is this obligation published?

[sqlite] Mozilla wiki 'avoid SQLite'

[Scott Hess]

For various reasons I've ended up as the SQLite rep w/in Chromium, and
I bookmarked that page awhile back to periodically revisit.  People
often seem to believe that SQLite magically solves tons of problems
with their persistence layer, without realizing that many of their
assumptions are based on things like having a DBA to take care of
things behind the scenes, or having a distinct database server process
probably running on pretty beefy hardware.

Also, to be honest, I find that developers often don't really _think_
about what they're doing.  SQLite essentially gives you a set of
b-trees with syntactic sugar over them, so if you cannot adequately
describe how your schema works in terms of raw b-trees, then SQLite
isn't going to be able to magically make things awesome behind your
back.  They also don't think about the evolution of their schema over
time.  So you see developers copy/paste some other code and make it
worse, and then never float things by an expert to see if they are
making poor decisions, and three years later someone has to
disentangle the horror.

Also, heaven defend me from people writing ad-hoc array-based query
systems that automatically generate bad SQL.  My experience with ORM
systems over the years has convinced me that if you spin up an entire
dedicated team to work really hard on that problem alone for years, it
will still be quite unsatisfying in the end.  One developer making a
bunch of stuff up and then abandoning it is just a recipe for later
pain.

There are many places where I think developers would have been better
served with a shared nosql-type store (like leveldb) mapping keys to
something a decent serialization system (like protocol buffers), and
which would have some well-defined and automated procedures for
handling things like transaction scheduling and recovery from
filesystem corruption and other errors.  It wouldn't be as good as
SQLite can be, but it wouldn't be as bad as the way many people use
SQLite in practice.

[To be fair to SQLite, much of my position on this isn't much
different than my position on MySQL in a past position.  SQL is a
challenge for many developers.]

-scott


On Sun, Jun 14, 2015 at 6:08 AM, Jean Chevalier  wrote:
> Somewhat contradictory the Mozilla Foundation being a member of the SQLite 
> Consortium while their performance wiki prominently features a warning to 
> developers against using SQLite allegedly for performance reasons. Guard me 
> from my friends...
>
> http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature
> ___
> sqlite-users mailing list
> sqlite-users at mailinglists.sqlite.org
> http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users

[sqlite] Mozilla wiki 'avoid SQLite'

[Scott Robison]

On Jun 14, 2015 9:43 AM, "Stephen Chrzanowski"  wrote:
>
> Most of that looks to be more like common sense things rather than SQLite
> specific, so why they're calling out SQLite, I've no idea.  Also, this doc
> was last modified more than a year ago.  Stuff has changed both within FF
> as well as SQlite since then.

They call out SQLite because that is the engine they embed in their
infrastructure. They aren't worried about people in their code base trying
to use MySQL, PgSQL, SQL Server, or Oracle. They're worried about their
developers using SQLite when some other format would be more appropriate
for some fuzzy value of "appropriate".

One shouldn't use a hammer on screws or a screwdriver on nails. Use the
right tool for the job. Sometimes FILE* based functionality is the right
tool. Sometimes not. It all comes down to the fuzzy concept of appropriate.