[sqlite] Mozilla wiki 'avoid SQLite'
Agreed. Two points. 1. Andl can never replace SQL in one step. The challenge is to find some useful niche and demonstrate an end-to-end solution to a problem, and show that as a total solution it's better than could be done with SQL. It should have a cross-platform front end UI/UX (say JavaScript, Java or Ruby but NOT C#/.NET) and (for now at least) a backend on SQLite, with Andl in the middle and NO ORM. 2. Despite what you might think, Andl is an easy language to learn. However, using it to do real work is a major challenge because the theoretic underpinnings are unfamiliar to most people. Actually solving problems using relations and operations on relations is new, even to very expert SQL users. There are many interesting problems that can be solved in just a few lines, but mentally constructing the steps to those few lines is tough. The time taken to write the Sudoku solver on the web site is way out of proportion to the length. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-boun...@mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of James K. Lowden Sent: Friday, 19 June 2015 7:11 AM To: sqlite-users at mailinglists.sqlite.org Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' On Wed, 17 Jun 2015 22:05:12 -0700 Darren Duncan wrote: > I also believe the world is ripe to have SQL alternatives, its just a > matter of ones appearing that are compelling to users for real work > and not just an academic exercise. The fact we're still generally with > SQL means this hasn't happened yet, but that doesn't mean it won't. Yes, I've been keeping track of TTM-inspired projects, and tried to convince Ingres to commercialize its D implementation. What makes David's andl unusual is that it's a new language atop a DBMS that is used in production. Still, I'm not so sure the world is ready for a better SQL. The evidence stands against, certainly. We already discarded one -- QUEL -- which hardly anyone remembers. A great deal of effort has gone into replacing SQL with less powerful constructs with no theoretical foundation, and into neutering the DBMS with ORM toys. Do not underestimate SQL's enormous inertia and network effect. Recognize that even if the syntax is better, engineering challenges remain if the promise of fidelity to the relational model is to be realized. The inertia stems from laziness and ignorance, never in short supply. A fraction of those who write SQL understand the math and logic underlying it. Forums like Stack Overflow are rife with questions that demonstrate as much; the answers are often not much better. If you're not thinking in terms of relational algebra and are unaware of the benefits of using logic to accomplish your task, changing syntaxes will only compound your problems. If you *are* thinking in those terms, it's still work to learn a new language. It's not clear that a better syntax -- if that's all you get -- would be seen as a worthwhile effort for very many people. The network effect is important, too. The market recognizes knowledge of SQL as a skill (even if it underestimates its value). That skill is portable across jobs and DBMS implementations. It is a way that programmers communicate to each other across the organization and time. A new language has to clear a threshold of "better" to be accepted. There are two engineering challenges that come to mind: performance and set-theory semantics. As David outlines in his blog, it's no mean feat to swap out SQL from any DBMS, including SQLite. Because the query language is assumed/known to be SQL, query planning, rewriting, transformation, and optimization is bound up in the grammar of SQL itself. Readers of this list appreciate how much work goes into that. Set-theory touches on the implementation, too. DISTINCT is implied in andl. It was left out of SQL because it requires a sort. The ramifications are nontrivial; consider joining on a nonkey and counting the results. Also remember that SQLite violates even SQL's set-theoretic features, such as when update T set X = X + 1 fails if X is declared unique. So I would say replacing SQL with andl or similar is in fact not analogous to replacing Perl with Python or C++ with Go. The requisite knowledge in the user community to appreciate the improvement is greater (and relatively nonexistent). And the engineering effort to build a DBMS is far more than to build a compiler. There's a reason Larry Ellison can affort a yacht almost 500 feet long. --jkl ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Mozilla wiki 'avoid SQLite'
Thank you for your comment. I take it seriously. Andl uses very few special characters: only about 3 or 4 that would require any explanation. But I take your point: a distinctive feature is that it lacks familiar keywords that draw the eye and help guide understanding. It makes it very compact for me to write samples and test programs, but not so good for new readers. Andl has close analogues to SELECT, WHERE, ORDER BY, GROUP BY, INSERT, UPDATE and DELETE, it already has JOIN, UNION, INTERSECT and MINUS (== EXCEPT) plus a number of other set and join variants, and it has no need for HAVING, ALL, DISTINCT, INTO and several others. It could use FROM, but this would add little. The syntax could easily be modified to use those (familiar) words, which might indeed make it more acceptable to SQL users. I have avoided doing so to this point to avoided confusing debates on another list. It might also be confusing, because although the analogues are there, the actual semantics are different, perhaps to the point of confusion. But it's worth thinking about. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Marc L. Allen Sent: Friday, 19 June 2015 1:10 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' Frankly, I've avoided taking a close look at Andl because of its massive use of special characters. I can usually work out a rough meaning on your examples because I know the SQL it came from. If I became 'Andl proficient' it probably wouldn't be a problem, but I gotta ask.. what's the issue with just using keywords? I feel like I'm looking at APL. -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of da...@andl.org Sent: Thursday, June 18, 2015 10:59 AM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' Yes, yours is the kind of situation Andl is aimed at. It does most of that stuff, but it's not ready for prime time yet. It's amazingly good at doing complex queries in just a few lines of code, but it lacks the external connections for it to be used for real applications. I don't really understand the 'row access' or 'multiple row sets' in terms of the need being filled, and what Andl has to do to meet it. Hopefully this will come out if and when people try to use it to do real work. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Marc L. Allen Sent: Thursday, 18 June 2015 11:51 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I don't understand all the issues with SQL as it is today. I doubt I'm at the level of most of the posters in this group. However, I do write a mess of it so I thought I'd give my two cents as a programmer. For what it does, SQL does it really well. That is, it takes a couple of sets of data, links them together and returns a set of data with given relations. But, that's just not enough. Recursive SQL is one of the best things to happen to SQL, though the syntax is rather wonky. But, I think it's done that way because SQL is NOT a programming language and we keep trying to turn it into one. Ok.. not turn it into one, per say, but we want to, quite often, do programming-like work on the data as part of the 'linking' process. This is especially prevalent in the systems supporting stored procedures. Personally, I find it a pain to combine SQL and programming in code. First, I have to generate the SQL, then I have to do whatever is necessary to iterate over the result set, often requiring additional SQL and additional iteration. So, if SPs are available, I tend to put a lot of logic in them and, frankly, SQL and it's variants really aren't good at that. Furthermore, the SQL environment is not part of the controlling programming environment and that causes additional problems. It's also slow to go back and forth, especially when the application and database are separated. I guess, at the end, is that I need a data language that has both set operations and iteration operations. I would want a language or construct that gives me row access during the operations so that I can finely tune the actions. Sort of an OnRow() function that works for all SQL operations. Also, the ability to split the incoming row set into multiple row sets for different purposes. Also... (insert weird data flow operation that doesn't exist natively). As I read back through this, I realize how poorly stated it is. But, I'm going to post it anyhow. Maybe someone will say, &quo
[sqlite] Mozilla wiki 'avoid SQLite'
Yes, yours is the kind of situation Andl is aimed at. It does most of that stuff, but it's not ready for prime time yet. It's amazingly good at doing complex queries in just a few lines of code, but it lacks the external connections for it to be used for real applications. I don't really understand the 'row access' or 'multiple row sets' in terms of the need being filled, and what Andl has to do to meet it. Hopefully this will come out if and when people try to use it to do real work. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Marc L. Allen Sent: Thursday, 18 June 2015 11:51 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I don't understand all the issues with SQL as it is today. I doubt I'm at the level of most of the posters in this group. However, I do write a mess of it so I thought I'd give my two cents as a programmer. For what it does, SQL does it really well. That is, it takes a couple of sets of data, links them together and returns a set of data with given relations. But, that's just not enough. Recursive SQL is one of the best things to happen to SQL, though the syntax is rather wonky. But, I think it's done that way because SQL is NOT a programming language and we keep trying to turn it into one. Ok.. not turn it into one, per say, but we want to, quite often, do programming-like work on the data as part of the 'linking' process. This is especially prevalent in the systems supporting stored procedures. Personally, I find it a pain to combine SQL and programming in code. First, I have to generate the SQL, then I have to do whatever is necessary to iterate over the result set, often requiring additional SQL and additional iteration. So, if SPs are available, I tend to put a lot of logic in them and, frankly, SQL and it's variants really aren't good at that. Furthermore, the SQL environment is not part of the controlling programming environment and that causes additional problems. It's also slow to go back and forth, especially when the application and database are separated. I guess, at the end, is that I need a data language that has both set operations and iteration operations. I would want a language or construct that gives me row access during the operations so that I can finely tune the actions. Sort of an OnRow() function that works for all SQL operations. Also, the ability to split the incoming row set into multiple row sets for different purposes. Also... (insert weird data flow operation that doesn't exist natively). As I read back through this, I realize how poorly stated it is. But, I'm going to post it anyhow. Maybe someone will say, "Hey, stupid... *this* exists.. try it." Heck, maybe Andl does it. I haven't looked. Marc -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of da...@andl.org Sent: Thursday, June 18, 2015 6:36 AM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I agree. It would be relatively easy to produce a new language with a syntax based on SQL, which was superficially familiar, but there are many necessary differences at the lower levels. One issue where bullet biting is needed is nulls and three-valued logic; another is duplicate rows and nameless or duplicate columns. The type system needs a complete overhaul too. C was a clean, polished and quite small language; SQL is none of those. And the biggest thing? Most SQL is used as a data sub-language, but the need is for a complete database programming language and a way out of the ORM mess. You can't get that from putting a layer on top of SQL. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Darren Duncan Sent: Thursday, 18 June 2015 5:29 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I disagree with the idea that a good SQL alternative would just be a superset of SQL as you propose. That has already been done numerous times, the principal manifestations being each SQL DBMS that has its own small or large differences in syntax and features from each other. SQL is already a very complex language due in part to most of its features each having their own custom syntax, often several variations per feature to boot, as well as lots of arbitrary limitations or specified inconsistent behaviors, a lot of these for keeping backwards compatibility with various old or vendor-specific ways of doing things. What a good SQL alternative would actually be is a
[sqlite] Mozilla wiki 'avoid SQLite'
> On Jun 18, 2015, at 11:11 PM, James K. Lowden > wrote: > > There's a reason Larry Ellison can affort a yacht almost 500 feet long. And an island to moor it. In any case, best luck to David and his project. He will need it.
[sqlite] Mozilla wiki 'avoid SQLite'
I agree. It would be relatively easy to produce a new language with a syntax based on SQL, which was superficially familiar, but there are many necessary differences at the lower levels. One issue where bullet biting is needed is nulls and three-valued logic; another is duplicate rows and nameless or duplicate columns. The type system needs a complete overhaul too. C was a clean, polished and quite small language; SQL is none of those. And the biggest thing? Most SQL is used as a data sub-language, but the need is for a complete database programming language and a way out of the ORM mess. You can't get that from putting a layer on top of SQL. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Darren Duncan Sent: Thursday, 18 June 2015 5:29 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I disagree with the idea that a good SQL alternative would just be a superset of SQL as you propose. That has already been done numerous times, the principal manifestations being each SQL DBMS that has its own small or large differences in syntax and features from each other. SQL is already a very complex language due in part to most of its features each having their own custom syntax, often several variations per feature to boot, as well as lots of arbitrary limitations or specified inconsistent behaviors, a lot of these for keeping backwards compatibility with various old or vendor-specific ways of doing things. What a good SQL alternative would actually be is a much more self-consistent and less redundant than SQL. It would still have all of SQL's expressive power and features so that any SQL code can be translated to it, including automatically, without too much circumlocution. That is how you would simplify the transition and re-utilization of existing code. The good alternative would actually be easier for a DBMS to implement also without losing any power. -- Darren Duncan On 2015-06-17 11:52 PM, ajm at zator.com wrote: > Indeed, I'm agree with Darren, and continuing its thought, perhaps that > hypothetical new language would be a clean extensi?n of SQL in the same way > that C++ was respect to C, simplifying the transition and reutilization of > legacy code. > > Cheers. > > -- > A.J. Millan >> >> Mensaje original >> De: >> Para: "'General Discussion of SQLite >> Database'" >> Fecha: Thu, 18 Jun 2015 14:50:40 +1000 >> Asunto: Re: [sqlite] Mozilla wiki 'avoid SQLite' >> >> The question for now is: does a new database programming language >> have a place? ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Mozilla wiki 'avoid SQLite'
> On Jun 18, 2015, at 5:10 PM, Marc L. Allen > wrote: > > Frankly, I've avoided taking a close look at Andl because of its massive use > of special characters. I can usually work out a rough meaning on your > examples because I know the SQL it came from. If I became 'Andl proficient' > it probably wouldn't be a problem, but I gotta ask.. what's the issue with > just using keywords? I feel like I'm looking at APL. mwahahaha! ?. my feeling exactly :P
[sqlite] Mozilla wiki 'avoid SQLite'
Simon Slavin wrote: > The thing I always found interesting about SQL was that it picks three > English words, INSERT, DELETE, UPDATE, and says that that's all you need to > do. And it's right ! Is there something special about the 'three-ness' of > database operations ? Or are you meant to think of it as two writing > operations (INSERT, DELETE) and a convenience operation which combines them > (UPDATE) ? If there was another word, what would it be ? REPLACE ? > DUPLICATE ? LDAP/X.500 has Add/Delete/Modify as well. (It also has Rename, which doesn't really make sense for a tabular data store, but is useful for a hierarchical data structure.) > Also, why is there only one English word needed for reading operations ? > What would a database language look like if it has more than one word ? > Would there be a difference between FIND and SCAN ? X.500 has three separate operations Read, List, and Search. (Read = retrieve contents of a single entry, List = list the names of the immediate children of an entry, Search = search for any entries matching a filter.) LDAP combined all of these functions into a single Search operation. It's often considered to be a mistake on the part of the LDAP designers. -- -- Howard Chu CTO, Symas Corp. http://www.symas.com Director, Highland Sun http://highlandsun.com/hyc/ Chief Architect, OpenLDAP http://www.openldap.org/project/
[sqlite] Mozilla wiki 'avoid SQLite'
On Wed, 17 Jun 2015 22:05:12 -0700 Darren Duncan wrote: > I also believe the world is ripe to have SQL alternatives, its just > a matter of ones appearing that are compelling to users for real work > and not just an academic exercise. The fact we're still generally > with SQL means this hasn't happened yet, but that doesn't mean it > won't. Yes, I've been keeping track of TTM-inspired projects, and tried to convince Ingres to commercialize its D implementation. What makes David's andl unusual is that it's a new language atop a DBMS that is used in production. Still, I'm not so sure the world is ready for a better SQL. The evidence stands against, certainly. We already discarded one -- QUEL -- which hardly anyone remembers. A great deal of effort has gone into replacing SQL with less powerful constructs with no theoretical foundation, and into neutering the DBMS with ORM toys. Do not underestimate SQL's enormous inertia and network effect. Recognize that even if the syntax is better, engineering challenges remain if the promise of fidelity to the relational model is to be realized. The inertia stems from laziness and ignorance, never in short supply. A fraction of those who write SQL understand the math and logic underlying it. Forums like Stack Overflow are rife with questions that demonstrate as much; the answers are often not much better. If you're not thinking in terms of relational algebra and are unaware of the benefits of using logic to accomplish your task, changing syntaxes will only compound your problems. If you *are* thinking in those terms, it's still work to learn a new language. It's not clear that a better syntax -- if that's all you get -- would be seen as a worthwhile effort for very many people. The network effect is important, too. The market recognizes knowledge of SQL as a skill (even if it underestimates its value). That skill is portable across jobs and DBMS implementations. It is a way that programmers communicate to each other across the organization and time. A new language has to clear a threshold of "better" to be accepted. There are two engineering challenges that come to mind: performance and set-theory semantics. As David outlines in his blog, it's no mean feat to swap out SQL from any DBMS, including SQLite. Because the query language is assumed/known to be SQL, query planning, rewriting, transformation, and optimization is bound up in the grammar of SQL itself. Readers of this list appreciate how much work goes into that. Set-theory touches on the implementation, too. DISTINCT is implied in andl. It was left out of SQL because it requires a sort. The ramifications are nontrivial; consider joining on a nonkey and counting the results. Also remember that SQLite violates even SQL's set-theoretic features, such as when update T set X = X + 1 fails if X is declared unique. So I would say replacing SQL with andl or similar is in fact not analogous to replacing Perl with Python or C++ with Go. The requisite knowledge in the user community to appreciate the improvement is greater (and relatively nonexistent). And the engineering effort to build a DBMS is far more than to build a compiler. There's a reason Larry Ellison can affort a yacht almost 500 feet long. --jkl
[sqlite] Mozilla wiki 'avoid SQLite'
Thanks for your response, James, and I agree with what you've said. My own language family, Muldis D, takes all of those things into account. For examples of this: 1. Muldis D is a general purpose language like for applications but it also is fundamentally savvy to the needs and features of relational and SQL databases (and other kinds of databases). Users of typical application languages would find all the features they're used to having and that they work in familiar ways. Users of typical SQL DBMSs would also find all the features they're used to having and that they work in familiar ways. 2. One killer feature of Muldis D is that one can use one language to work in both the application and database worlds. The long-standing wishes of many developers, to either use SQL features in applications or application language features in the database, will be satisfied. Not to mention the desire to have things just work when moving data between the two. Data types and business logic constraints and routines are all shareable. 3. Another killer feature of Muldis D is that one can losslessly translate any DBMS' version of SQL to it and have it work as they expect. So users' investment in SQL is preserved. Even if the Muldis D syntax isn't a superset of SQL, its feature set is, and so it just takes a translation layer for existing SQL-using applications to continue to work unchanged. This includes the power features of SQL like stored procedures and triggers and custom data types, not just lowest common denominator stuff like simple CRUD. I'm not going to harp on this too strongly as I still have to make Muldis D execute, but hopefully it will by the end of this summer. The reference implementation is stand-alone like a typical application programming language, but subsequent implementations will cross-compile as possible to existing SQL DBMSs so the work in those engines can be leveraged as possible. The reference is standalone so I am not limited by what current DBMSs support to make it work. -- Darren Duncan On 2015-06-18 2:11 PM, James K. Lowden wrote: > On Wed, 17 Jun 2015 22:05:12 -0700 > Darren Duncan wrote: > >> I also believe the world is ripe to have SQL alternatives, its just >> a matter of ones appearing that are compelling to users for real work >> and not just an academic exercise. The fact we're still generally >> with SQL means this hasn't happened yet, but that doesn't mean it >> won't. > > Yes, I've been keeping track of TTM-inspired projects, and tried to > convince Ingres to commercialize its D implementation. What makes > David's andl unusual is that it's a new language atop a DBMS that is > used in production. > > Still, I'm not so sure the world is ready for a better SQL. The > evidence stands against, certainly. We already discarded one -- QUEL -- > which hardly anyone remembers. A great deal of effort has gone into > replacing SQL with less powerful constructs with no theoretical > foundation, and into neutering the DBMS with ORM toys. > > Do not underestimate SQL's enormous inertia and network effect. > Recognize that even if the syntax is better, engineering > challenges remain if the promise of fidelity to the relational model is > to be realized. > > The inertia stems from laziness and ignorance, never in short supply. A > fraction of those who write SQL understand the math and logic > underlying it. Forums like Stack Overflow are rife with questions that > demonstrate as much; the answers are often not much better. If you're > not thinking in terms of relational algebra and are unaware of the > benefits of using logic to accomplish your task, changing syntaxes will > only compound your problems. If you *are* thinking in those terms, > it's still work to learn a new language. It's not clear that a better > syntax -- if that's all you get -- would be seen as a worthwhile effort > for very many people. > > The network effect is important, too. The market recognizes knowledge > of SQL as a skill (even if it underestimates its value). That skill is > portable across jobs and DBMS implementations. It is a way that > programmers communicate to each other across the organization and > time. A new language has to clear a threshold of "better" to be > accepted. > > There are two engineering challenges that come to mind: performance and > set-theory semantics. > > As David outlines in his blog, it's no mean feat to swap out SQL from > any DBMS, including SQLite. Because the query language is > assumed/known to be SQL, query planning, rewriting, transformation, and > optimization is bound up in the grammar of SQL itself. Readers of this > list appreciate how much work goes into that. > > Set-theory touches on the implementation, too. DISTINCT is > implied in andl. It was left out of SQL because it requires a sort. > The ramifications are nontrivial; consider joining on a nonkey and > counting the results. Also remember that SQLite
[sqlite] Mozilla wiki 'avoid SQLite'
I don't have enough experience in any FP language to be productive. C# has a lot of FP like features (lambdas, LINQ, etc) that I do know how to use. And ultimately, the C# version could be recoded in C++ if it's worth doing. The question for now is: does a new database programming language have a place? Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-boun...@mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of Drago, William @ CSG - NARDA-MITEQ Sent: Wednesday, 17 June 2015 11:58 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' > -Original Message- > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite- > users-bounces at mailinglists.sqlite.org] On Behalf Of david at andl.org > Sent: Monday, June 15, 2015 2:28 AM > To: 'General Discussion of SQLite Database' > Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' > > I won't abuse the patience of our hosts by prolonging this debate, but > I disagree strongly with this theme. > > I have almost certainly written more C/C++ code than you or most of > the people on this list, and I never choose it first. I am personally > at least 3 times as productive in C# as I am in C (slightly narrower > margin in C++), and computers are far cheaper than brains. If productivity is important is there any reason why you're not working in F#? I don't know that language myself, but I've seen some impressive demos. -- Bill Drago Staff Engineer L3 Narda-MITEQ 435 Moreland Road Hauppauge, NY 11788 631-272-5947 / William.Drago at L-3COM.com > > This theme is strongly reminiscent of arguments over moving from > assembly language, and it's basically wrong. The best tool is the one > that gets the required job done with maximal speed at minimal cost. > > And just for the record, C# does not compile into byte code. I suggest > you check your facts. > > Regards > David M Bennett FACS > > Andl - A New Database Language - andl.org > > -Original Message- > From: sqlite-users-bounces at mailinglists.sqlite.org > [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of > Aleksey Tulinov > Sent: Monday, 15 June 2015 10:32 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' > > On 15/06/15 01:00, Simon Slavin wrote: > > Simon, > > > Perhaps the next generation of computer languages will be designed > > by > computer, to let us speak to them in an efficient manner. > > > > I'm sure computer would insist on C, if not, then it's apparently a > software bug. > > On a serious note, i think it's rather question of programming > computers or programming another programs. As you've mentioned, higher > level languages often compiles into byte-code which is then > interpreted by virtual machine. > So you don't speak to machine, you speak to mediator who speak to > machine. > This by definition an overhead, with growing complexity of the > program, overhead will grow accordingly, this is unavoidable. > > I think it's also fair to say that SQL is not for programming machines > (no offense), it's for programming SQLite and other database > implementations. > Even if something is called "virtual machine", VM always behaves > somehow differently from The Machine, thus programmer' > efforts has mediated effect on latter. > > Of course VM could do a good job in a specific domain, but each VM > limits the liberty of expressing yourself to the machine and vice > versa. > > In my opinion best database language would reflect the way in which > database works and best computer programming language would reflect > the way in which computer works, as close as reasonably possible. > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users CONFIDENTIALITY, EXPORT CONTROL AND DISCLAIMER NOTE:This e-mail and any attachments are solely for the use of the addressee and may contain information that is privileged or confidential. Any disclosure, use or distribution of the information contained herein is prohibited. In the event this e-mail contains technical data within the definition of the International Traffic in Arms Regulations or Export Administration Regulations, it is subject to the export control laws of the U.S.Government. The recipient should check this e-mail and
[sqlite] Mozilla wiki 'avoid SQLite'
I should also clarify that I don't see SQL as *a* language but rather as a family of languages. Each actual SQL language in the family is whatever some particular SQL DBMS actually implements and how it behaves. So eg "SQLite SQL" is *a* SQL language, with generally a distinct version for each SQLite release. Then "SQLite SQL" is a considerably simpler language than say "Oracle SQL" or what have you. But I see this discussion about a SQL alternative to be relative to the SQL language family in general, which in my mind is its use case, rather than just SQLite specifically. That also means that the SQL alternative would in practice be a language family itself, assuming multiple implementations, though hopefully they would be a lot more consistent with each other than the SQL family languages have ended up being. -- Darren Duncan On 2015-06-18 1:27 PM, Darren Duncan wrote: > On 2015-06-18 3:35 AM, ajm at zator.com wrote: >> Darren: >> >> Maybe you're right. But to that relationship of "goodness" that would be >> desirable, perhaps we should add a few drops of pragmatism. Experience shows >> that in many cases, the perfect is the enemy of the good, and despite a >> disrupting transition (refusing the bad legacy of SQL) can avoid some >> drawbacks, however a non-disrupting evolution, has the advantage of being >> more >> easily accepted and assimilated by the users, as has happened before. > > I'm not proposing "perfect" at all, but rather I don't agree that the best > solution is to be perfectly backwards-compatible with SQL such that all SQL > code > is also valid code in the new language, as you are proposing. > > I don't think you realize just how complicated SQL as a whole is. What SQLite > implements is just a small subset of it. Just the core part of the ISO SQL > 9075-2:2011 language spec is about 1466 pages long and that leaves a lot out. > Besides this, various dozens SQL DBMS vendors have their own proprietary > additions or differences. Even just taking a subset an extending that, isn't > innovating beyond existing current practices, and we have dozens of those > variants. > > A lot of existing popular application languages have a much smaller language > complexity than SQL does. > > What I'm proposing as good is something more comparable in complexity to > those, > while still being able to express everything you can in SQL without any > greater > verbosity or difficulty in understanding. > > One primary way to achieve that is to represent nearly all operators / actions > using generic routine call syntax rather than custom syntax per operator, > which > is what languages like C, C++, C#, Java, etc do. For such as these, the > language itself is fairly small, and most of the details are just in the > libraries. SQL supports libraries too, eg stored procedures/functions/etc but > an inordinately large number of built-ins are not expressed in those terms. > > It is very reasonable to have generic syntax function calls to represent the > various parts of a SQL SELECT for example, similar to how we have array or set > or whatever routines in application languages today. That makes the syntax > relatively simple and user-extensible, and it by no means prevents a DBMS from > doing any query rewriting or optimizing or rearranging whatever it wants to do > to implement the over-all expression efficiently, in fact a DBMS may even have > an easier time of it. > > Don't get me wrong, a new language can have these qualities and still look > familiar to SQL users, its not like it means using APL etc. > > -- Darren Duncan > > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
[sqlite] Mozilla wiki 'avoid SQLite'
On 2015-06-18 3:35 AM, ajm at zator.com wrote: > Darren: > > Maybe you're right. But to that relationship of "goodness" that would be > desirable, perhaps we should add a few drops of pragmatism. Experience shows > that in many cases, the perfect is the enemy of the good, and despite a > disrupting transition (refusing the bad legacy of SQL) can avoid some > drawbacks, however a non-disrupting evolution, has the advantage of being > more easily accepted and assimilated by the users, as has happened before. I'm not proposing "perfect" at all, but rather I don't agree that the best solution is to be perfectly backwards-compatible with SQL such that all SQL code is also valid code in the new language, as you are proposing. I don't think you realize just how complicated SQL as a whole is. What SQLite implements is just a small subset of it. Just the core part of the ISO SQL 9075-2:2011 language spec is about 1466 pages long and that leaves a lot out. Besides this, various dozens SQL DBMS vendors have their own proprietary additions or differences. Even just taking a subset an extending that, isn't innovating beyond existing current practices, and we have dozens of those variants. A lot of existing popular application languages have a much smaller language complexity than SQL does. What I'm proposing as good is something more comparable in complexity to those, while still being able to express everything you can in SQL without any greater verbosity or difficulty in understanding. One primary way to achieve that is to represent nearly all operators / actions using generic routine call syntax rather than custom syntax per operator, which is what languages like C, C++, C#, Java, etc do. For such as these, the language itself is fairly small, and most of the details are just in the libraries. SQL supports libraries too, eg stored procedures/functions/etc but an inordinately large number of built-ins are not expressed in those terms. It is very reasonable to have generic syntax function calls to represent the various parts of a SQL SELECT for example, similar to how we have array or set or whatever routines in application languages today. That makes the syntax relatively simple and user-extensible, and it by no means prevents a DBMS from doing any query rewriting or optimizing or rearranging whatever it wants to do to implement the over-all expression efficiently, in fact a DBMS may even have an easier time of it. Don't get me wrong, a new language can have these qualities and still look familiar to SQL users, its not like it means using APL etc. -- Darren Duncan
[sqlite] Mozilla wiki 'avoid SQLite'
> Mensaje original > De: > Para: "'General Discussion of SQLite Database'" mailinglists.sqlite.org> > Fecha: Thu, 18 Jun 2015 14:50:40 +1000 > Asunto: Re: [sqlite] Mozilla wiki 'avoid SQLite' > Darren: Maybe you're right. But to that relationship of "goodness" that would be desirable, perhaps we should add a few drops of pragmatism. Experience shows that in many cases, the perfect is the enemy of the good, and despite a disrupting transition (refusing the bad legacy of SQL) can avoid some drawbacks, however a non-disrupting evolution, has the advantage of being more easily accepted and assimilated by the users, as has happened before. -- A.J. Millan
[sqlite] Mozilla wiki 'avoid SQLite'
Frankly, I've avoided taking a close look at Andl because of its massive use of special characters. I can usually work out a rough meaning on your examples because I know the SQL it came from. If I became 'Andl proficient' it probably wouldn't be a problem, but I gotta ask.. what's the issue with just using keywords? I feel like I'm looking at APL. -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of da...@andl.org Sent: Thursday, June 18, 2015 10:59 AM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' Yes, yours is the kind of situation Andl is aimed at. It does most of that stuff, but it's not ready for prime time yet. It's amazingly good at doing complex queries in just a few lines of code, but it lacks the external connections for it to be used for real applications. I don't really understand the 'row access' or 'multiple row sets' in terms of the need being filled, and what Andl has to do to meet it. Hopefully this will come out if and when people try to use it to do real work. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Marc L. Allen Sent: Thursday, 18 June 2015 11:51 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I don't understand all the issues with SQL as it is today. I doubt I'm at the level of most of the posters in this group. However, I do write a mess of it so I thought I'd give my two cents as a programmer. For what it does, SQL does it really well. That is, it takes a couple of sets of data, links them together and returns a set of data with given relations. But, that's just not enough. Recursive SQL is one of the best things to happen to SQL, though the syntax is rather wonky. But, I think it's done that way because SQL is NOT a programming language and we keep trying to turn it into one. Ok.. not turn it into one, per say, but we want to, quite often, do programming-like work on the data as part of the 'linking' process. This is especially prevalent in the systems supporting stored procedures. Personally, I find it a pain to combine SQL and programming in code. First, I have to generate the SQL, then I have to do whatever is necessary to iterate over the result set, often requiring additional SQL and additional iteration. So, if SPs are available, I tend to put a lot of logic in them and, frankly, SQL and it's variants really aren't good at that. Furthermore, the SQL environment is not part of the controlling programming environment and that causes additional problems. It's also slow to go back and forth, especially when the application and database are separated. I guess, at the end, is that I need a data language that has both set operations and iteration operations. I would want a language or construct that gives me row access during the operations so that I can finely tune the actions. Sort of an OnRow() function that works for all SQL operations. Also, the ability to split the incoming row set into multiple row sets for different purposes. Also... (insert weird data flow operation that doesn't exist natively). As I read back through this, I realize how poorly stated it is. But, I'm going to post it anyhow. Maybe someone will say, "Hey, stupid... *this* exists.. try it." Heck, maybe Andl does it. I haven't looked. Marc -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of da...@andl.org Sent: Thursday, June 18, 2015 6:36 AM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I agree. It would be relatively easy to produce a new language with a syntax based on SQL, which was superficially familiar, but there are many necessary differences at the lower levels. One issue where bullet biting is needed is nulls and three-valued logic; another is duplicate rows and nameless or duplicate columns. The type system needs a complete overhaul too. C was a clean, polished and quite small language; SQL is none of those. And the biggest thing? Most SQL is used as a data sub-language, but the need is for a complete database programming language and a way out of the ORM mess. You can't get that from putting a layer on top of SQL. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Darren Duncan Sent: Thursday, 18 June 2015 5:29 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I disagree wit
[sqlite] Mozilla wiki 'avoid SQLite'
Indeed, I'm agree with Darren, and continuing its thought, perhaps that hypothetical new language would be a clean extensi?n of SQL in the same way that C++ was respect to C, simplifying the transition and reutilization of legacy code. Cheers. -- A.J. Millan > > Mensaje original > De: > Para: "'General Discussion of SQLite Database'" mailinglistssqlite.org> > Fecha: Thu, 18 Jun 2015 14:50:40 +1000 > Asunto: Re: [sqlite] Mozilla wiki 'avoid SQLite' > >The question for now is: does a new database programming language have a >place? >
[sqlite] Mozilla wiki 'avoid SQLite'
I don't understand all the issues with SQL as it is today. I doubt I'm at the level of most of the posters in this group. However, I do write a mess of it so I thought I'd give my two cents as a programmer. For what it does, SQL does it really well. That is, it takes a couple of sets of data, links them together and returns a set of data with given relations. But, that's just not enough. Recursive SQL is one of the best things to happen to SQL, though the syntax is rather wonky. But, I think it's done that way because SQL is NOT a programming language and we keep trying to turn it into one. Ok.. not turn it into one, per say, but we want to, quite often, do programming-like work on the data as part of the 'linking' process. This is especially prevalent in the systems supporting stored procedures. Personally, I find it a pain to combine SQL and programming in code. First, I have to generate the SQL, then I have to do whatever is necessary to iterate over the result set, often requiring additional SQL and additional iteration. So, if SPs are available, I tend to put a lot of logic in them and, frankly, SQL and it's variants really aren't good at that. Furthermore, the SQL environment is not part of the controlling programming environment and that causes additional problems. It's also slow to go back and forth, especially when the application and database are separated. I guess, at the end, is that I need a data language that has both set operations and iteration operations. I would want a language or construct that gives me row access during the operations so that I can finely tune the actions. Sort of an OnRow() function that works for all SQL operations. Also, the ability to split the incoming row set into multiple row sets for different purposes. Also... (insert weird data flow operation that doesn't exist natively). As I read back through this, I realize how poorly stated it is. But, I'm going to post it anyhow. Maybe someone will say, "Hey, stupid... *this* exists.. try it." Heck, maybe Andl does it. I haven't looked. Marc -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of da...@andl.org Sent: Thursday, June 18, 2015 6:36 AM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I agree. It would be relatively easy to produce a new language with a syntax based on SQL, which was superficially familiar, but there are many necessary differences at the lower levels. One issue where bullet biting is needed is nulls and three-valued logic; another is duplicate rows and nameless or duplicate columns. The type system needs a complete overhaul too. C was a clean, polished and quite small language; SQL is none of those. And the biggest thing? Most SQL is used as a data sub-language, but the need is for a complete database programming language and a way out of the ORM mess. You can't get that from putting a layer on top of SQL. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Darren Duncan Sent: Thursday, 18 June 2015 5:29 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' I disagree with the idea that a good SQL alternative would just be a superset of SQL as you propose. That has already been done numerous times, the principal manifestations being each SQL DBMS that has its own small or large differences in syntax and features from each other. SQL is already a very complex language due in part to most of its features each having their own custom syntax, often several variations per feature to boot, as well as lots of arbitrary limitations or specified inconsistent behaviors, a lot of these for keeping backwards compatibility with various old or vendor-specific ways of doing things. What a good SQL alternative would actually be is a much more self-consistent and less redundant than SQL. It would still have all of SQL's expressive power and features so that any SQL code can be translated to it, including automatically, without too much circumlocution. That is how you would simplify the transition and re-utilization of existing code. The good alternative would actually be easier for a DBMS to implement also without losing any power. -- Darren Duncan On 2015-06-17 11:52 PM, ajm at zator.com wrote: > Indeed, I'm agree with Darren, and continuing its thought, perhaps that > hypothetical new language would be a clean extensi?n of SQL in the same way > that C++ was respect to C, simplifying the transition and reutilization of > legacy code. > > Cheers. > > -- > A.J. Millan >> >> Mensaje original >> De: >> Para:
[sqlite] Mozilla wiki 'avoid SQLite'
I disagree with the idea that a good SQL alternative would just be a superset of SQL as you propose. That has already been done numerous times, the principal manifestations being each SQL DBMS that has its own small or large differences in syntax and features from each other. SQL is already a very complex language due in part to most of its features each having their own custom syntax, often several variations per feature to boot, as well as lots of arbitrary limitations or specified inconsistent behaviors, a lot of these for keeping backwards compatibility with various old or vendor-specific ways of doing things. What a good SQL alternative would actually be is a much more self-consistent and less redundant than SQL. It would still have all of SQL's expressive power and features so that any SQL code can be translated to it, including automatically, without too much circumlocution. That is how you would simplify the transition and re-utilization of existing code. The good alternative would actually be easier for a DBMS to implement also without losing any power. -- Darren Duncan On 2015-06-17 11:52 PM, ajm at zator.com wrote: > Indeed, I'm agree with Darren, and continuing its thought, perhaps that > hypothetical new language would be a clean extensi?n of SQL in the same way > that C++ was respect to C, simplifying the transition and reutilization of > legacy code. > > Cheers. > > -- > A.J. Millan >> >> Mensaje original >> De: >> Para: "'General Discussion of SQLite Database'"> mailinglistssqlite.org> >> Fecha: Thu, 18 Jun 2015 14:50:40 +1000 >> Asunto: Re: [sqlite] Mozilla wiki 'avoid SQLite' >> >> The question for now is: does a new database programming language have a >> place?
[sqlite] Mozilla wiki 'avoid SQLite'
On 2015-06-17 9:50 PM, david at andl.org wrote: > The question for now is: does a new database programming language have a > place? When you ask the question as broadly as that, the answer is most definitely "yes". Just look at the wider world and you see there are dozens of application programming languages that have widespread use (and hundreds more that are more niche), and new ones that become widespread are appearing at a rate of around 1 a year on average, such as Swift and Go. The fact that these catch on means there are large numbers of people who think that there are worthwhile new languages, that don't just think we already have all the ones we need. I look at the database world in contrast, that is languages that are particularly savvy for the relational model like SQL, and I notice the world is largely passing it by in attempts to make a serious alternative. I also believe the world is ripe to have SQL alternatives, its just a matter of ones appearing that are compelling to users for real work and not just an academic exercise. The fact we're still generally with SQL means this hasn't happened yet, but that doesn't mean it won't. -- Darren Duncan
[sqlite] Mozilla wiki 'avoid SQLite'
I agree. SQL is quite deficient in terms of set-oriented updates. INSERT is more or less UNION, but UPDATE and DELETE have no set-oriented forms. The relational algebra describes operations on sets of tuples, where the only operation on attributes is to compare them by name or equal value. SQL implements most of the relational algebra directly, and all of it by combining operations. Updates should be semantically equivalent to an operation from the Relational Algebra followed by assignment (new value replaces old). INSERT works like that, but UPDATE and DELETE do not. Try writing a query to update all the salaries for a company where all the new salaries are found in some other table (ie a JOIN). Try writing a query to delete all the employees listed in some other table (another JOIN). Andl already has set-oriented UPDATE and DELETE, as well as the familiar predicate and computed styles. Thank you for asking. As it happens I have not been able to fully implement them on SQLite so far, because of limitations in the underlying SQL. Andl cannot perform any alterations on the columns of known tables, because that would change its relational type. It's easy to create a new table and copy data, but the issue of the proper way to handle versioning and migrations is still open. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-boun...@mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of Simon Slavin Sent: Wednesday, 17 June 2015 6:11 AM To: sqlite-users at sqlite.org; General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' On 16 Jun 2015, at 7:44pm, James K. Lowden wrote: > wrote: > >> What would make a database programming better, or best? > > Two things I've often pointed to are namespaces and regular > expressions. Another is compound datatypes. I don't have your problem with namespaces since, to me, they really are just prefixes. I do agree that regular expressions are a problem. They don't really belong in the language but they are very convenient when they are there. SQLite has JOINs (or sub-selects, which amount to the same thing at a low level) for INSERT and for SELECT but not for UPDATE. A few times when working with SQLite I've found myself writing UPDATE ... JOIN. And then having to do the job in my own code instead. And if you add JOIN to UPDATE you should probably add it to DELETE FROM too. The other problem with SQLite is the lack of ALTER TABLE ... DROP COLUMN. But to support it you need SQLite to have a proper internal model of which columns are used for what, rather than to just store and reparse the CREATE TABLE commands. The thing I always found interesting about SQL was that it picks three English words, INSERT, DELETE, UPDATE, and says that that's all you need to do. And it's right ! Is there something special about the 'three-ness' of database operations ? Or are you meant to think of it as two writing operations (INSERT, DELETE) and a convenience operation which combines them (UPDATE) ? If there was another word, what would it be ? REPLACE ? DUPLICATE ? Also, why is there only one English word needed for reading operations ? What would a database language look like if it has more than one word ? Would there be a difference between FIND and SCAN ? Simon. ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Mozilla wiki 'avoid SQLite'
> -Original Message- > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite- > users-bounces at mailinglists.sqlite.org] On Behalf Of david at andl.org > Sent: Monday, June 15, 2015 2:28 AM > To: 'General Discussion of SQLite Database' > Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' > > I won't abuse the patience of our hosts by prolonging this debate, but > I disagree strongly with this theme. > > I have almost certainly written more C/C++ code than you or most of the > people on this list, and I never choose it first. I am personally at > least 3 times as productive in C# as I am in C (slightly narrower > margin in C++), and computers are far cheaper than brains. If productivity is important is there any reason why you're not working in F#? I don't know that language myself, but I've seen some impressive demos. -- Bill Drago Staff Engineer L3 Narda-MITEQ 435 Moreland Road Hauppauge, NY 11788 631-272-5947 / William.Drago at L-3COM.com > > This theme is strongly reminiscent of arguments over moving from > assembly language, and it's basically wrong. The best tool is the one > that gets the required job done with maximal speed at minimal cost. > > And just for the record, C# does not compile into byte code. I suggest > you check your facts. > > Regards > David M Bennett FACS > > Andl - A New Database Language - andl.org > > -Original Message- > From: sqlite-users-bounces at mailinglists.sqlite.org > [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of > Aleksey Tulinov > Sent: Monday, 15 June 2015 10:32 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' > > On 15/06/15 01:00, Simon Slavin wrote: > > Simon, > > > Perhaps the next generation of computer languages will be designed by > computer, to let us speak to them in an efficient manner. > > > > I'm sure computer would insist on C, if not, then it's apparently a > software bug. > > On a serious note, i think it's rather question of programming > computers or programming another programs. As you've mentioned, higher > level languages often compiles into byte-code which is then interpreted > by virtual machine. > So you don't speak to machine, you speak to mediator who speak to > machine. > This by definition an overhead, with growing complexity of the program, > overhead will grow accordingly, this is unavoidable. > > I think it's also fair to say that SQL is not for programming machines > (no offense), it's for programming SQLite and other database > implementations. > Even if something is called "virtual machine", VM always behaves > somehow differently from The Machine, thus programmer' > efforts has mediated effect on latter. > > Of course VM could do a good job in a specific domain, but each VM > limits the liberty of expressing yourself to the machine and vice > versa. > > In my opinion best database language would reflect the way in which > database works and best computer programming language would reflect the > way in which computer works, as close as reasonably possible. > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users CONFIDENTIALITY, EXPORT CONTROL AND DISCLAIMER NOTE:This e-mail and any attachments are solely for the use of the addressee and may contain information that is privileged or confidential. Any disclosure, use or distribution of the information contained herein is prohibited. In the event this e-mail contains technical data within the definition of the International Traffic in Arms Regulations or Export Administration Regulations, it is subject to the export control laws of the U.S.Government. The recipient should check this e-mail and any attachments for the presence of viruses as L-3 does not accept any liability associated with the transmission of this e-mail. If you have received this communication in error, please notify the sender by reply e-mail and immediately delete this message and any attachments.
[sqlite] Mozilla wiki 'avoid SQLite'
On 17 Jun 2015, at 3:44am, Marc L. Allen wrote: > I don't know. Back in the day, assembly was low-level because it was directly > converted to machine code. C was high level because you could express more > complex structures without worrying about the underlying architecture. C was designed to be a processor-independent assembler code. C was called low-level because C pointers could be used to manipulate memory and the stack. C was called low-level because you had to understand the platform's memory map, stack and word size to use it, in contrast to FORTRAN/COBOL/BASIC where you blindly wrote your program with no understanding of your hardware. Thus, you get the argument between programmers about whether C was high-level or low-level. > I still like that distinction. I think people are trying to call C low level > simply because there are even higher level languages. It's not just 'high' or > 'low'. It's a spectrum. Characteristics, perhaps. Simon.
[sqlite] Mozilla wiki 'avoid SQLite'
Marco, >There's never been that kind of tone, the article has been written to >point >new Mozilla codebase contributors at possible pitfalls we already hit in >the past, and actually help them making informed decisions. That's not how one reads it. Start with only the title: "Performance/Avoid SQLite In Your Next Firefox Feature" Do you read with me? "If you want performance at any rate, AVOID SQLite ..." Then it starts enumerating pretended "SQLite pitfalls", pointing out several topics which are presented as inherent drawbacks to using SQLite from a FF pluggin. This is not "help them making their decision", just simply bashing the library which you seem to expect being almost reserved to core code. >It actually briefly explains in which cases a JSON log can work better Starting at ~1Mb data it advises storing stuff as compressed JSON. Now you're seriously trying to tell me that with a multi-Mb compressed complex JSON: -) locking the compressed JSON file -) reading it up in full, in memory -) unpacking it (with much more memory consumption, of course) -) parsing it (again more memory workspace needed) -) scanning the object tree in search of the data you're after -) reading and changing a single value in the tree -) flattening the whole tree back to JSON form -) repacking it -) rewriting the lot -) releasing the lock ALL of this sequence (in a multi-threaded environment with all the extra burden that comes with it) is going to take less time, waste less CPU cycles and use less RAM than a single SQLite UPDATE wrapped in a safe-to-use API, all in a smaller "footprint"? Seriously? And what does happen when two or more compressed JSON have to be used together to obtain/access the required information? Looks like a can or worms in this case. Note how the footprint argument is hypocritical since the SQLite library is already loaded and ready for work since it's much in use for the main thread, so the footprint is always there, that SQLite be used in pluggins or not. Any decently written safe wrapping API is ridiculously small and doesn't even count. I don't seem the only one to be utterly doubtful about this "compressed JSON > SQLite" assertion, when applied to the general case. >It tries to make people think before doing. Yes a simple flat file can be the best solution in simple cases but that shouldn't trigger the AVOID verb in general cases. And less simple flat files can fit the bill for another range of use cases. It would be wise to shape the arguments (a number of them being valid) in a different way. >It states to evaluate alternatives. Existing alternatives like OS.File >writeAtomic and JSON. Those are already used with success in both the >products and add-ons, when it makes sense. The last part of your sentence, "when it makes sense", clashes with the title (AVOID). Avoiding is not evaluating. >Sure the developer mus think to durability, backups, coherence, but >would it not be the same if he'd use SQLite? Definitely not, because you know that SQLite already takes greatest care of all the possible issues on countless platforms and this comes free: the code is already there, highly optimized and working. Code written by "average" pluggin developper can't compete with years of experience with those issues which are prone to overlook. Whenever a dark corner resulting in a bug in SQLite surfaces, the fix comes in no time and fixes all applications using SQLite at a time. > > After all, it's Mozilla devs themselves who designed pluggin APIs > and let > > "spurious main-thread SQL statements" be possible. If they were sooo > > clever, they would never had allowed that and they also would have > wrapped > > SQLite interface in a strictly limited set of rules enforced by a > safe API. > > That, they won't tell you. > >And they'd have a 100% bug free product with millions lines of code... No, >that's unrealistic. Great! So I've news for you: you can produce proven correct *-code-* using today's formal methods tools. Granted it would actually be irrealistic to dream of switching to a formal development process within months, even years, but such things do exist. Yet I never pretended that offering a safer API would solve all of the issues, just a number of those the text is about. > The Mozilla codebase comes from the 90s Netscape >codebase, at that time the most common thing was a single-threaded and >single-process browser, able to show text, some images and tables. The >reality evolved A LOT and the code had to evolve to cope with it. At the >time mozStorage (the SQLite wrapper) was written, there was still that >kind >of vision, and it was written as a main-thread synchronous API. Sure, now >we all know it was wrong, but at that time it was the right-ish thing to >do. The API grew a purely asynchronous alternative, but when you have >hundreds millions of users and thousands of add-ons using an API, you >can't >just say "sorry, we now break you
[sqlite] Mozilla wiki 'avoid SQLite'
ASFAIK also gcc uses some kind of intermediate rapresentation during compilation process... gimple (gcc -fdump-tree-all) and rtl (-dump-rtl-all) Il 16/giu/2015 22:40, "Simon Slavin" ha scritto: > Since this thread has ... become what it is ... I may as well add a couple > of details. > > The widely-used compiler LLVM uses an intermediate representation of your > code, called 'Bitcode'. In other words, it's not a simple case of > compiling straight to object code. This feeds into some distinctions made > between high level and low level languages in previous posts. > > I also wanted to comment on various things said about C by saying that > there is no industry agreement about whether C is a high level or low level > language. Mostly because those terms were never defined very clearly > because it was obvious to every programmer what they were looking at. If > the language included words which looked like human languages, it was a > high level language. If it was based around acronyms and very short words > like MOVE is was low level. C messed up the distinction and since then > we've been running to keep up. > > Simon. > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
[sqlite] Mozilla wiki 'avoid SQLite'
I don't know. Back in the day, assembly was low-level because it was directly converted to machine code. C was high level because you could express more complex structures without worrying about the underlying architecture. I still like that distinction. I think people are trying to call C low level simply because there are even higher level languages. It's not just 'high' or 'low'. It's a spectrum. C might be lower level to other languages, but higher level to assembler. > On Jun 16, 2015, at 4:40 PM, Simon Slavin wrote: > > Since this thread has ... become what it is ... I may as well add a couple of > details. > > The widely-used compiler LLVM uses an intermediate representation of your > code, called 'Bitcode'. In other words, it's not a simple case of compiling > straight to object code. This feeds into some distinctions made between high > level and low level languages in previous posts. > > I also wanted to comment on various things said about C by saying that there > is no industry agreement about whether C is a high level or low level > language. Mostly because those terms were never defined very clearly because > it was obvious to every programmer what they were looking at. If the > language included words which looked like human languages, it was a high > level language. If it was based around acronyms and very short words like > MOVE is was low level. C messed up the distinction and since then we've been > running to keep up. > > Simon. > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users This email and any attachments are only for use by the intended recipient(s) and may contain legally privileged, confidential, proprietary or otherwise private information. Any unauthorized use, reproduction, dissemination, distribution or other disclosure of the contents of this e-mail or its attachments is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original.
[sqlite] Mozilla wiki 'avoid SQLite'
Since this thread has ... become what it is ... I may as well add a couple of details. The widely-used compiler LLVM uses an intermediate representation of your code, called 'Bitcode'. In other words, it's not a simple case of compiling straight to object code. This feeds into some distinctions made between high level and low level languages in previous posts. I also wanted to comment on various things said about C by saying that there is no industry agreement about whether C is a high level or low level language. Mostly because those terms were never defined very clearly because it was obvious to every programmer what they were looking at. If the language included words which looked like human languages, it was a high level language. If it was based around acronyms and very short words like MOVE is was low level. C messed up the distinction and since then we've been running to keep up. Simon.
[sqlite] Mozilla wiki 'avoid SQLite'
On 16 Jun 2015, at 7:44pm, James K. Lowden wrote: > wrote: > >> What would make a database programming better, or best? > > Two things I've often pointed to are namespaces and regular > expressions. Another is compound datatypes. I don't have your problem with namespaces since, to me, they really are just prefixes. I do agree that regular expressions are a problem. They don't really belong in the language but they are very convenient when they are there. SQLite has JOINs (or sub-selects, which amount to the same thing at a low level) for INSERT and for SELECT but not for UPDATE. A few times when working with SQLite I've found myself writing UPDATE ... JOIN. And then having to do the job in my own code instead. And if you add JOIN to UPDATE you should probably add it to DELETE FROM too. The other problem with SQLite is the lack of ALTER TABLE ... DROP COLUMN. But to support it you need SQLite to have a proper internal model of which columns are used for what, rather than to just store and reparse the CREATE TABLE commands. The thing I always found interesting about SQL was that it picks three English words, INSERT, DELETE, UPDATE, and says that that's all you need to do. And it's right ! Is there something special about the 'three-ness' of database operations ? Or are you meant to think of it as two writing operations (INSERT, DELETE) and a convenience operation which combines them (UPDATE) ? If there was another word, what would it be ? REPLACE ? DUPLICATE ? Also, why is there only one English word needed for reading operations ? What would a database language look like if it has more than one word ? Would there be a difference between FIND and SCAN ? Simon.
[sqlite] Mozilla wiki 'avoid SQLite'
On 2015-06-16 11:44 AM, James K. Lowden wrote: > On Tue, 16 Jun 2015 09:56:38 +1000 wrote: > >> The question is: what should a database language do? Andl can already >> match or surpass SQL on database programming tasks, but is that >> interesting enough? >> >> What would make a database programming better, or best? > > Two things I've often pointed to are namespaces and regular > expressions. Another is compound datatypes. > > SQL and C both suffer from a single variable namespace. We get around > it by using prefixes, e.g., "local_memcpy" or "annualized_returns". > > C++ added namespaces to the language. I suggest SQL's successor do the > same, but use the Unix filesystem's hierarchical namespace as a model. > Putatively, SQL in general actually DOES have namespaces in practice, though they are fixed-depth rather than variable-depth. Depending on the DBMS, you have the "catalog" level, the "schema" level, the Oracle "package" level, and then your functions and tables etc have otherwise unqualified names beneath those. Depending on the DBMS, each of those levels may exist or not, but the number of levels is fixed, that part unlike the Unix filesystem. I think the SQL standard specifies catalog/schema/object itself. For my part with my database-savvy Muldis D programming language (that I hope to have executing within a few months), namespaces for both routines and relations (tables) etc are arbitrary/variable depth like you propose, like either the Unix file system or like programming languages such as Perl or C# or others. Note that Muldis D and Andl have some influences in common, but David beat me to market as it were with an executable. -- Darren Duncan
[sqlite] Mozilla wiki 'avoid SQLite'
p.s. The spellchecker just kindly advised that maybe changing the name of "andl" might help reduce future unfortunate typos / "corrections". :-) -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Jonathan Moules Sent: Tuesday, June 16, 2015 4:33 PM To: 'General Discussion of SQLite Database' Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' > The question is: what should a database language do? Andl can already match > or surpass SQL on database programming tasks, but is that interesting enough? As much as anything, that depends on what problem you're targeting, and even your audience. At the risk of rekindling the High/low/assembly level discussion, certainly at the high level, languages generally all have different design goals, and because of this they appeal to different people. This list appears to have a very high proportion of computer science types, so we see discussions about recursions and "syntactic sugar" and all that jazz; conversely, ask on a list for web-developers and you'll get a very different set of answers. Personally, as someone whose SQL-fu is weak, and who isn't a computer scientist, one of the best things about SQL is that it's English-like and there's a very low barrier to entry. Consider this statement, which is about as complex as 90% of my SQL ever gets: SELECT * from buildings where height > 30 and colour = 'mauve' With just 2 minutes of explaining I could probably get my mum to understand what was going on there. Ok, that's an unrealistically low bar, but many people who use SQL just have simple queries/problems. While I appreciate andl doesn't have documentation yet, it doesn't look like it will pass the "not a computer scientist" test for usability. Just my 2c. Cheers, Jonathan -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of da...@andl.org Sent: Tuesday, June 16, 2015 12:57 AM To: ajm at zator.com; 'General Discussion of SQLite Database' Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' >>>I think the best database language should reflect how earthlings >>>think about the data, and the best computer programming language would reflect easily the result we want get from them. Care to expand on that? I'm developing a new database language: Andl. My starting point has been the relational model, Codd-Date-Darwen and The Third Manifesto. My (only) competitor seems to be SQL, which has a 40+ year lead. Nothing like a challenge! The question is: what should a database language do? Andl can already match or surpass SQL on database programming tasks, but is that interesting enough? What would make a database programming better, or best? Regards David M Bennett FACS Andl - A New Database Language - andl.org ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users This message has been scanned for viruses by MailControl - www.mailcontrol.com Click https://www.mailcontrol.com/sr/43VYmEOYO7bGX2PQPOmvUj!GOBh06pKK8EdQhM6i4FvmoG0cFV2y0WTEr3dfsj1m6uKlINFphbL3KeH4!zzvzA== to report this email as spam. HR Wallingford and its subsidiaries uses faxes and emails for confidential and legally privileged business communications. They do not of themselves create legal commitments. Disclosure to parties other than addressees requires our specific consent. We are not liable for unauthorised disclosures nor reliance upon them. If you have received this message in error please advise us immediately and destroy all copies of it. HR Wallingford Limited Howbery Park, Wallingford, Oxfordshire, OX10 8BA, United Kingdom Registered in England No. 02562099 ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users HR Wallingford and its subsidiaries uses faxes and emails for confidential and legally privileged business communications. They do not of themselves create legal commitments. Disclosure to parties other than addressees requires our specific consent. We are not liable for unauthorised disclosures nor reliance upon them. If you have received this message in error please advise us immediately and destroy all copies of it. HR Wallingford Limited Howbery Park, Wallingford, Oxfordshire, OX10 8BA, United Kingdom Registered in England No. 02562099
[sqlite] Mozilla wiki 'avoid SQLite'
> The question is: what should a database language do? Andl can already match > or surpass SQL on database programming tasks, but is that interesting enough? As much as anything, that depends on what problem you're targeting, and even your audience. At the risk of rekindling the High/low/assembly level discussion, certainly at the high level, languages generally all have different design goals, and because of this they appeal to different people. This list appears to have a very high proportion of computer science types, so we see discussions about recursions and "syntactic sugar" and all that jazz; conversely, ask on a list for web-developers and you'll get a very different set of answers. Personally, as someone whose SQL-fu is weak, and who isn't a computer scientist, one of the best things about SQL is that it's English-like and there's a very low barrier to entry. Consider this statement, which is about as complex as 90% of my SQL ever gets: SELECT * from buildings where height > 30 and colour = 'mauve' With just 2 minutes of explaining I could probably get my mum to understand what was going on there. Ok, that's an unrealistically low bar, but many people who use SQL just have simple queries/problems. While I appreciate andl doesn't have documentation yet, it doesn't look like it will pass the "not a computer scientist" test for usability. Just my 2c. Cheers, Jonathan -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of da...@andl.org Sent: Tuesday, June 16, 2015 12:57 AM To: ajm at zator.com; 'General Discussion of SQLite Database' Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' >>>I think the best database language should reflect how earthlings >>>think about the data, and the best computer programming language would reflect easily the result we want get from them. Care to expand on that? I'm developing a new database language: Andl. My starting point has been the relational model, Codd-Date-Darwen and The Third Manifesto. My (only) competitor seems to be SQL, which has a 40+ year lead. Nothing like a challenge! The question is: what should a database language do? Andl can already match or surpass SQL on database programming tasks, but is that interesting enough? What would make a database programming better, or best? Regards David M Bennett FACS Andl - A New Database Language - andl.org ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users This message has been scanned for viruses by MailControl - www.mailcontrol.com Click https://www.mailcontrol.com/sr/43VYmEOYO7bGX2PQPOmvUj!GOBh06pKK8EdQhM6i4FvmoG0cFV2y0WTEr3dfsj1m6uKlINFphbL3KeH4!zzvzA== to report this email as spam. HR Wallingford and its subsidiaries uses faxes and emails for confidential and legally privileged business communications. They do not of themselves create legal commitments. Disclosure to parties other than addressees requires our specific consent. We are not liable for unauthorised disclosures nor reliance upon them. If you have received this message in error please advise us immediately and destroy all copies of it. HR Wallingford Limited Howbery Park, Wallingford, Oxfordshire, OX10 8BA, United Kingdom Registered in England No. 02562099
[sqlite] Mozilla wiki 'avoid SQLite'
On Tue, 16 Jun 2015 09:56:38 +1000 wrote: > The question is: what should a database language do? Andl can already > match or surpass SQL on database programming tasks, but is that > interesting enough? > > What would make a database programming better, or best? Two things I've often pointed to are namespaces and regular expressions. Another is compound datatypes. SQL and C both suffer from a single variable namespace. We get around it by using prefixes, e.g., "local_memcpy" or "annualized_returns". C++ added namespaces to the language. I suggest SQL's successor do the same, but use the Unix filesystem's hierarchical namespace as a model. Putatively, ATTACH DATABASE 'foo.db' as /db/local/foo; CHANGE DATABASE /db/local/foo; CREATE TABLE annualized/returns ; As far as I can tell, all the basic file and link management features of the filesystem have analogous utility in a database. (I would extend that idea to permission bits, about which we could have a robust discussion if you're interested.) Regular expressions likewise belong in a query language. The LIKE operator, an NIH relic of SQL's IBM origins, belongs on the ash heap of history. Best to follow Russ Cox's advice and restrict the regex syntax to constructs with O(n) complexity. Finally, compound datatypes would simplify and encourage the use of natural keys. Something along these lines, CREATE UDT stock_key ( asof datetime, cusip char(8) ); CREATE TABLE prices( stock_key, price float ); CREATE TABLE returns( days int, return float, stock_key references prices ); Constraints defined on the compound user-defined type would of course apply to whatever table it appears in. I thought I'd pass these along because you asked and because I don't remember seeing them in TTM. I assume you're supporting row-generators. Do you intend to support table-comparison, too? What about insert/update as assignment? --jkl
[sqlite] Mozilla wiki 'avoid SQLite'
On Tue, 16 Jun 2015 10:36:45 +0200
Marco Bonardo wrote:
> The contents of the page came out from actual bugs and misuses we hit
> in years of use and experience with it and were discussed with
> attention. Most also have workarounds or suggested fixes.
While I'm sure that's true, the page is filled with dubious
assertions.
Almost everything it says about I/O is evidence-free. The implications
of a file (or the database) using multiple extents on the disk
("fragmentation") is the best example. For an application like FF, how
much data, and how many seeks, are we really talking about? How many of
them are in name only, because the OS reads whole tracks and buffers
hundreds of megabytes in the page cache?
Is fsync "very expensive"? Not likely. Compared to network latency,
it's near zero. On many consumer hard disks, it *is* zero, because the
drive lies about completed writes.
Do small, simple writes perform better in JSON? If so, by how much?
What complexity then leaks back into the application to manage deletes
and index seeks, no to mention concurrency?
The page doesn't mention the value of constraints for correctness. It
doesn't mention the convenience to the developer of being able to
inspect and query the data outside the application.
Yes, sure, right tool for the job. In deciding, best to know what the
tool does, and what the job is.
--jkl
[sqlite] Mozilla wiki 'avoid SQLite'
2015-06-16 1:36 GMT+02:00 Jean-Christophe Deschamps : > While I can agree with some of the most obvious "don't be dumb" remarks, > there are many things that can't be let unchallenged. And I don't > appreciate the overall tone: "WE at MozillaBigBalls are all clever enough > to use SQLite smartly but you mere mortals are not, so don't even think to > use that fragile piece of softawre." > There's never been that kind of tone, the article has been written to point new Mozilla codebase contributors at possible pitfalls we already hit in the past, and actually help them making informed decisions. It's not intended for global consumption. Sure it's public, cause there's no reason to "hide" technical document that might be useful to contributors, just to complicate their access. Can you find any page that states Mozilla doesn't like SQLite or wants anyone to abandon it? It is used everywhere, from mercurial repositories to Firefox OS... > In it's introduction, the blurb talks about SQLite hidden complexity. Then > it avdises Mozilla pluggin developers to avoid SQLite at almost any rate, > due to "performance reasons", and recommends compressed JSON/logs instead. > It actually briefly explains in which cases a JSON log can work better and at the end it also states what to do and avoid if still SQLite is found to be the best store (and it clearly happens!). It tries to make people think before doing. > Yet this guy(s) advocate that *-every-* pluggin devs should independently > roll his own storage layer It states to evaluate alternatives. Existing alternatives like OS.File writeAtomic and JSON. Those are already used with success in both the products and add-ons, when it makes sense. Sure the developer mus think to durability, backups, coherence, but would it not be the same if he'd use SQLite? You must find the right compromise between durability and performance, you still have to handle corruption cases. Thinking about that stuff is part of normal planning for any feature, independently from the chosen store. > Then another question remains: instead of putting the onus on SQLite being > huge (footprint) and slow (CPU, the 22s "example"), why don't the author(s) > of the blurb question the real root causes of the evil they condemn and > openly recognize that the problem lies entirely elsewhere, perhaps in > Mozilla core code design itself? > Because it's clear the article is about SQLite use in the Mozilla codebase. It's clear there are issues in the Mozilla codebase itself (I'd be surprised of the opposite!). > After all, it's Mozilla devs themselves who designed pluggin APIs and let > "spurious main-thread SQL statements" be possible. If they were sooo > clever, they would never had allowed that and they also would have wrapped > SQLite interface in a strictly limited set of rules enforced by a safe API. > That, they won't tell you. > And they'd have a 100% bug free product with millions lines of code... No, that's unrealistic. The Mozilla codebase comes from the 90s Netscape codebase, at that time the most common thing was a single-threaded and single-process browser, able to show text, some images and tables. The reality evolved A LOT and the code had to evolve to cope with it. At the time mozStorage (the SQLite wrapper) was written, there was still that kind of vision, and it was written as a main-thread synchronous API. Sure, now we all know it was wrong, but at that time it was the right-ish thing to do. The API grew a purely asynchronous alternative, but when you have hundreds millions of users and thousands of add-ons using an API, you can't just say "sorry, we now break you all". You must play fairly. Sure, the Firefox add-ons are the most powerful add-ons around, they can do anything, and while this might be (as you point out) a downside, cause they can perform poorly, it is also their major selling point. There are still a lots of things you can do with a Firefox add-on that you can't do in any other browser add-on. Also, this has nothing to do with the fact SQLite can still be the wrong choice for certain data store needs (as it's the best for others), that is still the main purpose of that article. > Also, if Mozilla devs were sooo much more clever than average Joe and sooo > caring about performance, they certainly would have fixed the hundreds of > memory leaks that plague FF users > This is going off topic, I'd be happy to digress about this but it's not the right place. If you have suggestions or bugs, the codebase is open and the bug tracker is public, you can reach every single developer through mail or IRC. You have all the tools to make the difference. Marco
[sqlite] Mozilla wiki 'avoid SQLite'
With a relational model you have a choice between relational calculus or relational algebra. SQL does one of them. I have used a language which did the other, long ago in a land far away. On Tue, Jun 16, 2015 at 9:56 AM, wrote: > >>>I think the best database language should reflect how earthlings think > about the data, and the best computer programming language would reflect > easily the result we want get from them. > > Care to expand on that? > > I'm developing a new database language: Andl. My starting point has been > the > relational model, Codd-Date-Darwen and The Third Manifesto. My (only) > competitor seems to be SQL, which has a 40+ year lead. Nothing like a > challenge! > > The question is: what should a database language do? Andl can already match > or surpass SQL on database programming tasks, but is that interesting > enough? > > What would make a database programming better, or best? > > Regards > David M Bennett FACS > > Andl - A New Database Language - andl.org > > > > > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > -- Christopher Vance
[sqlite] Mozilla wiki 'avoid SQLite'
2015-06-15 18:05 GMT+02:00 Jean Chevalier : > What I'm thinking now is to what extent the developer who put up the > Mozilla wiki page was entitled to put up opinions and statements either > expressed or implied about a third-party product on behalf of the Mozilla > Foundation (though is that the same Foundation that pays the Consortium > member fee?). The mozilla wiki contains informations useful to contributors to the mozilla codebase and more generally "mozillians", it's not intended to be a global resource of information like wikipedia, nor any kind of official documentation on how to use libraries. It's targeted to code written against the Mozilla codebase and it's not even in the official MDN. The document you pointed at was created some time ago, cause we had a problem with developers taking "the easy way" too often, when they needed a store they just used SQLite (or better mozStorage, our wrapper) because the API was nice and already available, without doing any kind of analysis of their needs. The title was (likely) chosen explicitly negative to make people read the article and clarify the point before going deep into details. And it helped, now people ask what's the best store for their kind of workflow, and clearly SQLite is still a used option. Please don't attach any kind of negative bias to a wiki article, it's just a technical article about possible pitfalls our developers will hit (and have hit) if they don't think what they are doing. Nothing more than that. SQLite is used extensively in every single Mozilla product, so what? . The contents of the page came out from actual bugs and misuses we hit in years of use and experience with it and were discussed with attention. Most also have workarounds or suggested fixes. Honestly I just think you are giving too much importance to a technical document with a clear target and very well expressed points, rather than to the fact SQLite is happily used everywhere. It should just be used properly, not randomly.
[sqlite] Mozilla wiki 'avoid SQLite'
>>>I think the best database language should reflect how earthlings think about the data, and the best computer programming language would reflect easily the result we want get from them. Care to expand on that? I'm developing a new database language: Andl. My starting point has been the relational model, Codd-Date-Darwen and The Third Manifesto. My (only) competitor seems to be SQL, which has a 40+ year lead. Nothing like a challenge! The question is: what should a database language do? Andl can already match or surpass SQL on database programming tasks, but is that interesting enough? What would make a database programming better, or best? Regards David M Bennett FACS Andl - A New Database Language - andl.org
[sqlite] Mozilla wiki 'avoid SQLite'
In my tool box, I have a tack hammer, a claw hammer and a nail gun. I broke my fingernail with the tack hammer. I still have the bandaid on my thumb from the claw hammer. Now I just have to read the instruction manual for the nail gun
[sqlite] Mozilla wiki 'avoid SQLite'
I've refrained to comment about the OP linked page but I can't resist that long. I won't enter the C, C--, C++, C#, Java, Python, COBOL, Basic, assembler, Ruby, YouNameIt sub-debate. I found the rant on MozillaWiki way too wrong on too many points to keep quiet. While I can agree with some of the most obvious "don't be dumb" remarks, there are many things that can't be let unchallenged. And I don't appreciate the overall tone: "WE at MozillaBigBalls are all clever enough to use SQLite smartly but you mere mortals are not, so don't even think to use that fragile piece of softawre." In it's introduction, the blurb talks about SQLite hidden complexity. Then it avdises Mozilla pluggin developers to avoid SQLite at almost any rate, due to "performance reasons", and recommends compressed JSON/logs instead. That's very odd, to say the least. All regular readers of this list have seen a number of threads where the intricacies of how hard it is to guarantee both some concurrency and ACID properties down to magnetic surface or chips' gates is difficult in the general case. You all know that SQLite actually uses all possible steps to bring this guarantee the closest possible to reality. Yet this guy(s) advocate that *-every-* pluggin devs should independently roll his own storage layer mostly from scratch, while pointing out the unstabilityor uncertainty associated with mobile devices processes/OSes. That is nothing but calling for (1) huge number of new entertaining bugs (2) at best, duplicate efforts to make that work in a highly multi-threaded environment where concurrency will need to be dealt with with greatest care. No, the "average" pluggin dev won't be able to come up in minutes with a rock solid storage layer portable to a myriad of platforms. And this is precisely what SQLite has been thru over years. As has already been pointed out, storing largish private data in compressed JSON or log file(s) will be (beyond often being a greater resource hog) a real nightmare when several tabs/windows will possibly need to concurrently read/modify/write stuff in there. Writing rants about SQLite being a resource hog and a performance breaker is one thing, guiding towards a clean way to replace it in practice is another (mystical) beast, at least for the average pluggin developper. Then another question remains: instead of putting the onus on SQLite being huge (footprint) and slow (CPU, the 22s "example"), why don't the author(s) of the blurb question the real root causes of the evil they condemn and openly recognize that the problem lies entirely elsewhere, perhaps in Mozilla core code design itself? They almost silently agree that allowing main-thread SQL access is a huge no-no. They recognize that allowing unlimited use of various (possibly conflicting) pragmas, random schemas and/or statements could harm. But what those hypocritical guys don't tell you is that the root cause is in the cahotic spiralling development history of Mozilla. After all, it's Mozilla devs themselves who designed pluggin APIs and let "spurious main-thread SQL statements" be possible. If they were sooo clever, they would never had allowed that and they also would have wrapped SQLite interface in a strictly limited set of rules enforced by a safe API. That, they won't tell you. Also, if Mozilla devs were sooo much more clever than average Joe and sooo caring about performance, they certainly would have fixed the hundreds of memory leaks that plague FF users (at least on Windows and for almost a decade) and they would have "spring cleaned" their messy codebase so that one can't see JS fragments kept running after the FF tabs which launched it was closed and blatant bugs like these. Watch memory bytes slowy growing toward 2Gb while FF is left "idle" and the RJ45 unplugged, without any pluggin set up... Is that the fault of pluggin devs, Dr. Hipp laziness, or the result of their own careless work? With today's FF W7 x64 (ditto for x86) release, you can't let it have, say, 30 tabs open 24/7 on "passive" forum pages (no ads, no sound, no video, nothing dynamic) for more than 2-3 weeks on the average. Then the machine gets so slow and unresponsive that you can only kill FF and restart it. At the time it has reopen and refreshed all the tabs, you witness its memory bytes now sit at 600 Mb (compared to ~2Gb before), with everything as functional as before killing it. Is that also SQLite fault or is that a sad joke? These are things that you don't see with either IE or Chrome (each of them having their own drawbacks too). In short and beyond a few trivial advises, the authors of this rant are either surprisingly ignorant or utterly hypocritical, but unfair at any rate. I've nothing against Mozilla per se and I sincerely acknowledge that maintaining such a huge open product is very hard, but this page is simply plain wrong.
[sqlite] Mozilla wiki 'avoid SQLite'
Am Sun, 14 Jun 2015 21:01:33 -0400 schrieb Stephen Chrzanowski: > Its like going back to Visual Basic and including the > interpreter in your "compiled" application with your source code basically > copy/pasted right into the final EXE output (Encrypted/obfuscated/whatever). Off topic indeed, but JFTR: Visual Basic never worked that way. Wolfgang
[sqlite] Mozilla wiki 'avoid SQLite'
Ditto. My C# code routinely gets close to native C performance whenever I get around to benchmarking it, which is not often these days. And I can actually write code that runs safely on my 8 cores or on my teraflop GPU if I really need the speed. But as I said, I really don't think this is the place. I hate seeing posts like that pass without response but I think I've said more than enough on the subject. The hazards of https://xkcd.com/386/ are ever-present. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Alex Bowden Sent: Monday, 15 June 2015 7:46 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' Oh dear. So you think that a high level language is one that does things by calling a lower level language? Stop embarrassing yourself. A high level language is one where the language designers are free to use whatever structural concepts best fit the problems that the language is designed to address, whereas a low level language is one where the language designers are restricted to using structural concepts that map directly onto an assumed hardware model. The assumption that a low level language compiles to assembler is nonsense. I don?t remember any of them doing this before C, and even later, many compilers still compile directly to machine code. Assuming that higher level languages compile to lower level languages is also nonsense. And even for ones that do, and a classic example of a language designed to do this would be Ratfor, then provided that the lower level language is compiling to machine code, then the only thing that is being slowed is the build process. Not runtime execution. Now 20 years ago, folk like you, but better informed, would have argued that well written lower level language code was more efficient than higher level language code. And sometimes they would have been correct. But note the qualification of well written. And even then, this was debatable. If a language provides efficient higher level data structures, or fine grain synchronisation of multiple threads, then the user is more likely to use them than if they have to create them themselves for each platform. However today, it is rarely true. Good modern optimisers can achieve efficiency that few, if any, low level language programmers will ever achieve. And the thing is, that those compilers can do far broader optimisation of higher level languages that don?t expose features like the ability to write to a random address. The optimisation potential of C, beyond very local optimisation, is minimal. I?ve been running the same little benchmark across every hardware and language platform that I?ve used, for over 30 years. And I confess, I was very surprised when Java passed C in about 2005. And yes, that was Java compiled to byte codes with runtime JIT compilation of the byte codes to machine code. But it was faster. The trend hasn?t reversed. > On 14 Jun 2015, at 23:42, Scott Doctor wrote: > > On 6/14/2015 3:00 PM, Simon Slavin wrote: >> The result is that that higher level the language you write in, the better. > > I disagree. The use of languages higher than C result in slow bloated code. A > language that calls a language that calls a language. Simple programs become > multi-megabyte resource hogs. I agree that C compilers are able to optimize > assembler code to a level that hand-coded assembler probably could not > achieve. The problem is that higher level languages are not compiling to > assembler, but compiling to the language below it. > > > Scott Doctor > scott at scottdoctor.com > -- > > > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Mozilla wiki 'avoid SQLite'
As I said, I won't fuel the fire. This is the wrong place for this kind of debate, and you should realise that. Your post reflects immaturity and narrow experience, but I won't be the one to broaden your outlook. Your post contains provable errors of fact, but I won't be correcting them. The subject at hand is SQLite and the super-high-level language SQL, and it would be discourteous of us to forget it. Discussion closed. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-boun...@mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of Scott Doctor Sent: Monday, 15 June 2015 5:03 PM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' On 6/14/2015 11:28 PM, david at andl.org wrote: > I won't abuse the patience of our hosts by prolonging this debate, but > I disagree strongly with this theme. So you disagree with a disagreement? > I have almost certainly written more C/C++ code than you or most of > the people on this list, I doubt it. > and I never choose it first. I am personally at least 3 times as > productive in C# as I am in C (slightly narrower margin in C++), and > computers are far cheaper than brains. C# is just a bastardized version of C++ which is a bastardized version of C. If you only program in C# then I guess you have only written code for PC's. A whole other world exists beyond PC's and Microsoft. C# is in no way portable, neither is C++. Only C is truly portable. Both C++ and C# require committing to a specific compiler product which is the antithesis of C. I am often forced to use C++ as the cross compiler platforms (such as C# or Embarcadero (Borland) compiler) force such to use the system GUI, but all of "My" code, as compared to the GUI code, that does the real work is written in C. C++ is merely a wrapper around the C language. C and C++ co-mingle very nicely. > This theme is strongly reminiscent of arguments over moving from > assembly language, and it's basically wrong. Well a bunch of very experienced programmers, with very diverse backgrounds seem to disagree. > The best tool is the one that gets the required job done with maximal > speed at minimal cost. Which is more important: How fast you can crank out code with minimal effort (which means you are letting others write the canned code portion of your code), or creating something new where you do the hard work so the end user has a well designed efficient product? > And just for the record, C# does not compile into byte code. I suggest > you check your facts. Quite believable. Which is why Microsoft software is so efficient, fast, small, and lacking of bugs. Scott Doctor scott at scottdoctor.com -- ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Mozilla wiki 'avoid SQLite'
> Mensaje original > De: Scott Hess > Para: General Discussion of SQLite Database mailinglists.sqlite.org> > Fecha: Sun, 14 Jun 2015 10:10:24 -0700 > Asunto: Re: [sqlite] Mozilla wiki 'avoid SQLite' > While certainly, the thread has derived from the concept of the OP, and even some think that this kind of discussion is out of place, I must confess that I enjoy greatly with this type of thread, that seem to me like images of certain books that seem too dry without them. Usually I learn a lot from them, because readed dispassionately, the contributions of the various comments -which by the way tend to be fairly level- often clarify and put in context, aspects that are important. From the allegation of the Mozilla Wiki, to the response of Dr. Hipp, passing through the funny misidentification of Alex Bowden (who does not make mistakes?). That said, I would like to complete the opinion of Aleksey Tulinov: >In my opinion best database language would reflect the way in which >database works and best computer programming language would reflect the >way in which computer works, as close as reasonably possible. I think the best database language should reflect how earthlings think about the data, and the best computer programming language would reflect easily the result we want get from them. Cheers. -- A. J. Millan
[sqlite] Mozilla wiki 'avoid SQLite'
What I'm thinking now is to what extent the developer who put up the Mozilla wiki page was entitled to put up opinions and statements either expressed or implied about a third-party product on behalf of the Mozilla Foundation (though is that the same Foundation that pays the Consortium member fee?).? Everyone is an individual and no one is free of bias, but I'm not sure we can always foretell when we're about to say something, the consequences of our actions.? Like, does everyone of us here represent SQLite? As to the uninformed passer-by it may not be clear who among us could be a product developer and who isn't. Thereby the importance of good behaviour (but what is good behaviour and does it always have to appear to be mild and proper, and in no way possibly lead to misunderstanding, accusation, or offence, even when certain things are hard to tell?) Perhaps I should have questioned the developer who wrote the page instead of the Foundation, or his lack of confronting other developers who in the past had misused SQLite instead of the product, or making it clear that he was referring to them (but then again delicate balance, being his colleague), or the circumstances that lead to him being able to put up the page without any dissenting voices annotating and qualifying its content.? Or perhaps there were dissenting voices and they were quelled, or perhaps there weren't and there was just disinterest, in which case he should be praised for breaking the ice and actually posting some content where some was needed.? Or perhaps I wasn't brave enough to drop him a line and chose instead to bring it up here.? I'm afraid we might never know, in this as elsewhere, it's all history in the making, and for as long as we're humans, it's all imperfect. I suppose I did it from the viewpoint of someone who being new to SQLite had visited the web pages of Consortium Members thinking that they could be better qualified than most to give feedback on the product. Had I visited the wikis of various consortium members to form an opinion and found criticism on all or most of them, I would have walked away from SQLite based on their feedback and not give it a second thought. At least, based on how I interpreted the page to be though not the page verbatim. As it happens, I already held an opinion about SQLite, I didn't need Mozilla to tell me, and it didn't make me change my mind, but I thought that new visitors could have been affected, that's why I thought I had to say something. Then as the conversation changed to talking about C/C++/C#, I feel a bit like Yuko, the girl in Battle Royale who by mistake poisons the wrong person, which by some unpredictable domino effect leads to everyone around the kitchen be killed. Quoting Linus and trying to reassure myself into thinking that heated exchange is all right, as long as we believe in ourselves and each other, and believe that method and passion will help bring up the truth, kind of helped, though I'm not sure it was enough to restore my inner peace. He said: "Because if you want me to "act professional", I can tell you that I'm not interested. I'm sitting in my home office wearing a bathrobe. The same way I'm not going to start wearing ties, I'm *also* not going to buy into the fake politeness, the lying, the office politics and backstabbing, the passive aggressiveness, and the buzzwords. Because THAT is what "acting professionally" results in: people resort to all kinds of really nasty things because they are forced to act out their normal urges in unnatural ways." And by God, have I seen plenty of that in places where I've worked? Coming from people of all ranks. J.B. Nicholson-Owens wrote: | | Jean Chevalier wrote: | > | > Somewhat contradictory the Mozilla Foundation being a member of the | > SQLite Consortium while their performance wiki prominently features | > a warning to developers against using SQLite allegedly for | > performance reasons. Guard me from my friends... | > | > http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature | | That page describes situations where SQLite doesn't give best available | performance, reduce complexity, or provide needed features, and then | goes on to justify those conclusions (including recognizing how SQLite | isn't unique in these respects: "This isn't an indictment of SQLite | itself -- any other relational embedded DB would pose the same | challenges."). I'm not sure what point you were raising in your post, | so I'll have to guess. Your summary suggests that you expected SQLite | Consortium members will endorse SQLite even in situations where SQLite | isn't a good fit due to preferring other tradeoffs. | | Are SQLite Consortium members somehow obliged to endorse SQLite for | tasks even when other approaches present more desirable tradeoffs? | If so, where is this obligation published? |
[sqlite] Mozilla wiki 'avoid SQLite'
I won't abuse the patience of our hosts by prolonging this debate, but I disagree strongly with this theme. I have almost certainly written more C/C++ code than you or most of the people on this list, and I never choose it first. I am personally at least 3 times as productive in C# as I am in C (slightly narrower margin in C++), and computers are far cheaper than brains. This theme is strongly reminiscent of arguments over moving from assembly language, and it's basically wrong. The best tool is the one that gets the required job done with maximal speed at minimal cost. And just for the record, C# does not compile into byte code. I suggest you check your facts. Regards David M Bennett FACS Andl - A New Database Language - andl.org -Original Message- From: sqlite-users-boun...@mailinglists.sqlite.org [mailto:sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of Aleksey Tulinov Sent: Monday, 15 June 2015 10:32 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' On 15/06/15 01:00, Simon Slavin wrote: Simon, > Perhaps the next generation of computer languages will be designed by computer, to let us speak to them in an efficient manner. > I'm sure computer would insist on C, if not, then it's apparently a software bug. On a serious note, i think it's rather question of programming computers or programming another programs. As you've mentioned, higher level languages often compiles into byte-code which is then interpreted by virtual machine. So you don't speak to machine, you speak to mediator who speak to machine. This by definition an overhead, with growing complexity of the program, overhead will grow accordingly, this is unavoidable. I think it's also fair to say that SQL is not for programming machines (no offense), it's for programming SQLite and other database implementations. Even if something is called "virtual machine", VM always behaves somehow differently from The Machine, thus programmer' efforts has mediated effect on latter. Of course VM could do a good job in a specific domain, but each VM limits the liberty of expressing yourself to the machine and vice versa. In my opinion best database language would reflect the way in which database works and best computer programming language would reflect the way in which computer works, as close as reasonably possible. ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Mozilla wiki 'avoid SQLite'
> On 15 Jun 2015, at 15:44, Scott Robison wrote: > > Too many of these technology based discussions (whether languages or > operating systems or text editors or database engines or whatever) break > down into almost a religious fervor of "this is the one true You mean like your comment from this morning? "The use of languages higher than C result in slow bloated code. A language that calls a language that calls a language. Simple programs become multi-megabyte resource hogs.? Frame it. Put it over your bed. And use it to remind you to put your brain in gear, before opening your mouth.
[sqlite] Mozilla wiki 'avoid SQLite'
On Mon, Jun 15, 2015 at 9:08 AM, Marc L. Allen wrote: > Perhaps you might want to consider an apology and check your headers next > time. > > I'm pretty sure the comment you quoted from this morning was written by > Scott Doctor, not Scott Robinson. > Thanks for the reply to the list. I'd replied privately previously to point it out to him, and he apologized off list in kind. It would have been a "great" zinger if only all Scotts were equivalent. ;) > > -Original Message- > From: sqlite-users-bounces at mailinglists.sqlite.org [mailto: > sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of Alex Bowden > Sent: Monday, June 15, 2015 11:00 AM > To: General Discussion of SQLite Database > Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' > > > > On 15 Jun 2015, at 15:44, Scott Robison wrote: > > > > Too many of these technology based discussions (whether languages or > > operating systems or text editors or database engines or whatever) > > break down into almost a religious fervor of "this is the one true > > You mean like your comment from this morning? > > "The use of languages higher than C result in slow bloated code. A > language that calls a language that calls a language. Simple programs > become multi-megabyte resource hogs.? > > Frame it. Put it over your bed. And use it to remind you to put your > brain in gear, before opening your mouth. > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > > > > This email and any attachments are only for use by the intended > recipient(s) and may contain legally privileged, confidential, proprietary > or otherwise private information. Any unauthorized use, reproduction, > dissemination, distribution or other disclosure of the contents of this > e-mail or its attachments is strictly prohibited. If you have received this > email in error, please notify the sender immediately and delete the > original. > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users > -- Scott Robison
[sqlite] Mozilla wiki 'avoid SQLite'
Oh dear. So you think that a high level language is one that does things by calling a lower level language? Stop embarrassing yourself. A high level language is one where the language designers are free to use whatever structural concepts best fit the problems that the language is designed to address, whereas a low level language is one where the language designers are restricted to using structural concepts that map directly onto an assumed hardware model. The assumption that a low level language compiles to assembler is nonsense. I don?t remember any of them doing this before C, and even later, many compilers still compile directly to machine code. Assuming that higher level languages compile to lower level languages is also nonsense. And even for ones that do, and a classic example of a language designed to do this would be Ratfor, then provided that the lower level language is compiling to machine code, then the only thing that is being slowed is the build process. Not runtime execution. Now 20 years ago, folk like you, but better informed, would have argued that well written lower level language code was more efficient than higher level language code. And sometimes they would have been correct. But note the qualification of well written. And even then, this was debatable. If a language provides efficient higher level data structures, or fine grain synchronisation of multiple threads, then the user is more likely to use them than if they have to create them themselves for each platform. However today, it is rarely true. Good modern optimisers can achieve efficiency that few, if any, low level language programmers will ever achieve. And the thing is, that those compilers can do far broader optimisation of higher level languages that don?t expose features like the ability to write to a random address. The optimisation potential of C, beyond very local optimisation, is minimal. I?ve been running the same little benchmark across every hardware and language platform that I?ve used, for over 30 years. And I confess, I was very surprised when Java passed C in about 2005. And yes, that was Java compiled to byte codes with runtime JIT compilation of the byte codes to machine code. But it was faster. The trend hasn?t reversed. > On 14 Jun 2015, at 23:42, Scott Doctor wrote: > > On 6/14/2015 3:00 PM, Simon Slavin wrote: >> The result is that that higher level the language you write in, the better. > > I disagree. The use of languages higher than C result in slow bloated code. A > language that calls a language that calls a language. Simple programs become > multi-megabyte resource hogs. I agree that C compilers are able to optimize > assembler code to a level that hand-coded assembler probably could not > achieve. The problem is that higher level languages are not compiling to > assembler, but compiling to the language below it. > > > Scott Doctor > scott at scottdoctor.com > -- > > > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Mozilla wiki 'avoid SQLite'
Perhaps you might want to consider an apology and check your headers next time. I'm pretty sure the comment you quoted from this morning was written by Scott Doctor, not Scott Robinson. -Original Message- From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:sqlite-users-boun...@mailinglists.sqlite.org] On Behalf Of Alex Bowden Sent: Monday, June 15, 2015 11:00 AM To: General Discussion of SQLite Database Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite' > On 15 Jun 2015, at 15:44, Scott Robison wrote: > > Too many of these technology based discussions (whether languages or > operating systems or text editors or database engines or whatever) > break down into almost a religious fervor of "this is the one true You mean like your comment from this morning? "The use of languages higher than C result in slow bloated code. A language that calls a language that calls a language. Simple programs become multi-megabyte resource hogs.? Frame it. Put it over your bed. And use it to remind you to put your brain in gear, before opening your mouth. ___ sqlite-users mailing list sqlite-users at mailinglists.sqlite.org http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users This email and any attachments are only for use by the intended recipient(s) and may contain legally privileged, confidential, proprietary or otherwise private information. Any unauthorized use, reproduction, dissemination, distribution or other disclosure of the contents of this e-mail or its attachments is strictly prohibited. If you have received this email in error, please notify the sender immediately and delete the original.
[sqlite] Mozilla wiki 'avoid SQLite'
This is not an answer to just the latest post, it is directed at multiple
posts.
1. C# is typically compiled to Common Intermediate Language which very much
is a form of "byte code" in that it is not directly executed by the
processor. The .net environment will just in time compile that into a
machine representation, or you can ahead of time compile that into a native
image. Still, CIL is involved, and while it is not Java Byte Code, it is a
byte code.
2. C is closer to "portable" than other high level languages because it
doesn't attempt to do as much. It doesn't have the huge run time
environment of .net or Java. However, at some point, any library code you
use, if available on more than one platform, even if written in C, will
begin to take advantage of the preprocessor or separate source modules so
as to smooth over imperfect portability.
3. C++ is just as "portable" as C. I can write C++ that runs on multiple
platforms and uses multiple compilers. It was designed to run well on the
same types of platforms where C runs well. It has the same difficulties
with cross platform as C, needing to use different source files or
conditionalized code via the preprocessor.
3a. I'll grant that C++ does not have as many shipping compilers in the
world as does C, so in that sense C is more portable. It is easier to write
a C compiler than a C++ compiler. And that different C++ compilers support
the language to greater and lesser extents, so it is not as portable. That
being said, the reason C is so portable is that a lot of people ignore
language features that came after C89/C90 and that rely on implementation
defined behavior (or use the preprocessor). I just don't encounter C99 or
C11 code much at all in the wild.
4. Even modern CPUs aren't directly executing all of the machine code our
compilers generate. Those instructions often get re-written internally to
various microcode instructions. From that perspective, our CPUs JIT
"compile" the lowest level machine code we can provide into something even
better for them.
5. The more important aspect of efficiency is not whether your language
compiles to machine code or some byte code or uses object oriented
abstractions or goes for straight procedural code. The more important thing
is using efficient algorithms and data structures. A smart algorithm on a
very high level run time byte code interpreted language (say Python) *can*
beat the pants off a pedestrian algorithm written in finely tuned C.
I like C, and C++, and C#, and Python, and JavaScript, and a bunch of other
languages. Each have their place. I *prefer* C++ with certain libraries as
it gives me a good blend of low level capabilities of C with the higher
level abstractions available from an object oriented language and the
availability of much functionality in huge frameworks like C# & Java.
Too many of these technology based discussions (whether languages or
operating systems or text editors or database engines or whatever) break
down into almost a religious fervor of "this is the one true {fill in the
blank}". Our job as programmers or developers or engineers (pick your
label) is to express things in a form that the computer can process.
SDR
On Mon, Jun 15, 2015 at 5:26 AM, wrote:
> Ditto. My C# code routinely gets close to native C performance whenever I
> get around to benchmarking it, which is not often these days. And I can
> actually write code that runs safely on my 8 cores or on my teraflop GPU if
> I really need the speed.
>
> But as I said, I really don't think this is the place. I hate seeing posts
> like that pass without response but I think I've said more than enough on
> the subject. The hazards of https://xkcd.com/386/ are ever-present.
>
> Regards
> David M Bennett FACS
>
> Andl - A New Database Language - andl.org
>
> -Original Message-
> From: sqlite-users-bounces at mailinglists.sqlite.org [mailto:
> sqlite-users-bounces at mailinglists.sqlite.org] On Behalf Of Alex Bowden
> Sent: Monday, 15 June 2015 7:46 PM
> To: General Discussion of SQLite Database
> Subject: Re: [sqlite] Mozilla wiki 'avoid SQLite'
>
> Oh dear. So you think that a high level language is one that does things
> by calling a lower level language? Stop embarrassing yourself.
>
> A high level language is one where the language designers are free to use
> whatever structural concepts best fit the problems that the language is
> designed to address, whereas a low level language is one where the
> language designers are restricted to using structural concepts that map
> directly onto an assumed hardware model.
>
> The assumption that a low level language compiles to assembler is
> nonsense. I don?t remember any of them doing this before C, and even
> later, many compilers still compile directly to machine code.
>
> Assuming that higher l
[sqlite] Mozilla wiki 'avoid SQLite'
On 15/06/15 01:00, Simon Slavin wrote: Simon, > Perhaps the next generation of computer languages will be designed by > computer, to let us speak to them in an efficient manner. > I'm sure computer would insist on C, if not, then it's apparently a software bug. On a serious note, i think it's rather question of programming computers or programming another programs. As you've mentioned, higher level languages often compiles into byte-code which is then interpreted by virtual machine. So you don't speak to machine, you speak to mediator who speak to machine. This by definition an overhead, with growing complexity of the program, overhead will grow accordingly, this is unavoidable. I think it's also fair to say that SQL is not for programming machines (no offense), it's for programming SQLite and other database implementations. Even if something is called "virtual machine", VM always behaves somehow differently from The Machine, thus programmer' efforts has mediated effect on latter. Of course VM could do a good job in a specific domain, but each VM limits the liberty of expressing yourself to the machine and vice versa. In my opinion best database language would reflect the way in which database works and best computer programming language would reflect the way in which computer works, as close as reasonably possible.
[sqlite] Mozilla wiki 'avoid SQLite'
On 6/14/2015 11:28 PM, david at andl.org wrote: > I won't abuse the patience of our hosts by prolonging this debate, but I > disagree strongly with this theme. So you disagree with a disagreement? > I have almost certainly written more C/C++ code than you or most of the > people on this list, I doubt it. > and I never choose it first. I am personally at least 3 > times as productive in C# as I am in C (slightly narrower margin in C++), > and computers are far cheaper than brains. C# is just a bastardized version of C++ which is a bastardized version of C. If you only program in C# then I guess you have only written code for PC's. A whole other world exists beyond PC's and Microsoft. C# is in no way portable, neither is C++. Only C is truly portable. Both C++ and C# require committing to a specific compiler product which is the antithesis of C. I am often forced to use C++ as the cross compiler platforms (such as C# or Embarcadero (Borland) compiler) force such to use the system GUI, but all of "My" code, as compared to the GUI code, that does the real work is written in C. C++ is merely a wrapper around the C language. C and C++ co-mingle very nicely. > This theme is strongly reminiscent of arguments over moving from assembly > language, and it's basically wrong. Well a bunch of very experienced programmers, with very diverse backgrounds seem to disagree. > The best tool is the one that gets the > required job done with maximal speed at minimal cost. Which is more important: How fast you can crank out code with minimal effort (which means you are letting others write the canned code portion of your code), or creating something new where you do the hard work so the end user has a well designed efficient product? > And just for the record, C# does not compile into byte code. I suggest you > check your facts. Quite believable. Which is why Microsoft software is so efficient, fast, small, and lacking of bugs. Scott Doctor scott at scottdoctor.com --
[sqlite] Mozilla wiki 'avoid SQLite'
On 14 Jun 2015, at 9:19pm, Richard Hipp wrote: > There was a huge debate in > the software industry in the 1980s about whether applications should > be coded in assembly or C. Hand-coded assembly is theoretically > faster than machine-code generated by a C compiler (or at least it was > in the 80s - that point is debatable now). But in practice, programs > written in C tended to perform better than those written in assembly. > The reason is that C being higher level enabled programmers to spend > less time fiddling with bits and more time developing better > algorithms. To extend this a bit, current compilers are even better. They're not really compilers any more. They include compilation, interpretation, and simulation at multiple levels. The program is analysed as a directed graph of changed variables. Knowledge of the target processor is used to figure out optimal ways to use the various resources (registers, bandwidth, cache sizes). It's ridiculous how good these 'compilers' are. The result is that that higher level the language you write in, the better. The language the programmer is writing is how they tell the compiler what they want done. The more expressive the language, the better the communication between programmer and compiler. Writing in assembly language just shifts this burden onto a human, and computers are much better at keeping track of all these things than humans are. The result is that the advantage which used to result from writing in assembler is blown away by what can be achieved by a good compiler of a good high-level programming language. Some of Dr Hipp's recent work with the revised query planner reminds me of the above. Rather than simply analyzing the SQL command, SQLite now looks not only at the command and the chunkiness of the data (the results of ANALYZE) but also at how the command combines the various things SQL can express, trying to guess what the programmer is really trying to do. Rather than the old days of parsing, what's going on is moving up to the level of semantic analysis. At this stage we have finally got to a level where alternatives to SQL -- either relational or not -- can be quantitively analyzed. The 'best language' will be the one that compiles to the fastest-executing virtual machine program. I have no idea what the result of this will be. Perhaps the next generation of computer languages will be designed by computer, to let us speak to them in an efficient manner. Simon.
[sqlite] Mozilla wiki 'avoid SQLite'
I as well disagree that "the higher language, the better". The bloat .NET adds to the application size for both processing, memory and drive usage is astronomical. Its like going back to Visual Basic and including the interpreter in your "compiled" application with your source code basically copy/pasted right into the final EXE output (Encrypted/obfuscated/whatever). I don't write in .NET, but I do agree for a developer, it CAN be a lot easier with everything that language has to offer. The libraries and functions and procedures and mechanisms and whatever else included to make our jobs easier are numerous. But optimization goes out the window when your 'compiler' does nothing more than translate your code at run-time instead of converting it into byte-code the OS will understand and deal with more directly. Everything has to eventually work its way down to byte-code, but including the overhead of translating the language at run-time doesn't help in the least. On Sun, Jun 14, 2015 at 6:42 PM, Scott Doctor wrote: > On 6/14/2015 3:00 PM, Simon Slavin wrote: > >> The result is that that higher level the language you write in, the >> better. >> > > I disagree. The use of languages higher than C result in slow bloated > code. A language that calls a language that calls a language. Simple > programs become multi-megabyte resource hogs. I agree that C compilers are > able to optimize assembler code to a level that hand-coded assembler > probably could not achieve. The problem is that higher level languages are > not compiling to assembler, but compiling to the language below it. > > > Scott Doctor > scott at scottdoctor.com > -- > > > > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
[sqlite] Mozilla wiki 'avoid SQLite'
Bottom line point, the general "feeling" of that page is to dissuade people from even CONSIDERING using SQLite for future enhancements (Considering the age of that page). I won't get on board with that. What I meant by "common sense" is more towards the hardware the software is going to be used on. A desktop PC running Win7 with 32k or 64k cluster sizes via NTFS on a platter drive of any RPM is going to operate a LOT differently than an 4k to 8k cluster sized SD card formatted either at FAT32 or EXT3 on a phone or tablet, when we're talking about the nuts and bolts and welds that hold the machine, OS, and software together. Different latencies between OS and storage medium (Forget the software making the calls at this point) SHOULD drive the "optimization" development towards "platform particular" configurations. None of that has anything to do with SQLite in itself. And when SQLite does enter the picture for optimization, a really basic check is to look at the OS the software is running on to find out what kind of file system it might be writing to, then create/regenerate the database paging system based on a "best guess" cluster size, or have a pre-built matrix to indicate what 'normal' drive configurations are and build the database against what that matrix says works well. That is only ONE aspect that can SERIOUSLY decrease IO waits. Basically, my impression of most of that page, which the information contained within is absolutely correct IMO, is that they're targeting mostly the wrong foundation. I know that it is a page of "Things to watch for" but targeting SQLite itself.. well.. I feel that the author is looking through the wrong looking glass. On Sun, Jun 14, 2015 at 11:57 AM, Scott Robison wrote: > On Jun 14, 2015 9:43 AM, "Stephen Chrzanowski" > wrote: > > > > Most of that looks to be more like common sense things rather than SQLite > > specific, so why they're calling out SQLite, I've no idea. Also, this > doc > > was last modified more than a year ago. Stuff has changed both within FF > > as well as SQlite since then. > > They call out SQLite because that is the engine they embed in their > infrastructure. They aren't worried about people in their code base trying > to use MySQL, PgSQL, SQL Server, or Oracle. They're worried about their > developers using SQLite when some other format would be more appropriate > for some fuzzy value of "appropriate". > > One shouldn't use a hammer on screws or a screwdriver on nails. Use the > right tool for the job. Sometimes FILE* based functionality is the right > tool. Sometimes not. It all comes down to the fuzzy concept of appropriate. > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
[sqlite] Mozilla wiki 'avoid SQLite'
On Sun, Jun 14, 2015 at 1:19 PM, Richard Hipp wrote: > On 6/14/15, Scott Hess wrote: >> SQLite essentially gives you a set of >> b-trees with syntactic sugar over them, > > SQL (and I speak in general terms here, not just of SQLite) provides > way more than syntactic sugar over b-trees. The syntactic sugar over > b-trees part is the *least* of the many advantages of SQL. Other more > important features include: To be clear, what I mean by this is that if you have a set of operations which you could do in your native language, and you decide to instead push them into SQLite, and you are concerned about performance and memory use, then it really behooves you to be able to describe how to accomplish those operations using SQLite's basic data structures. Many developers build their SQL assumptions in an environment with a memory-constrained app server calling out to a database server running on the beefiest machine their ops people can acquire. The set of things-that-are-appropriate are going to be different in an environment where "Map a multi-gigabyte index into memory" is a valid approach. In case it isn't clear from my post (I have a bit of a head cold going on, so I probably shouldn't be writing posts on public mailing lists), IMHO SQLite has done a really solid job covering the bases for the area of operations it has staked out, and has done a good job of not adding features inappropriate to those goals. It's just that IMHO developers often make dubious assumptions about what _else_ SQLite can do. -scott
[sqlite] Mozilla wiki 'avoid SQLite'
On 14 Jun 2015, at 4:43pm, Stephen Chrzanowski wrote: > Most of that looks to be more like common sense things rather than SQLite > specific, so why they're calling out SQLite, I've no idea. Because the SQLite engine is part of FireFox, and SQLite calls are very easy to do from parts of, and extensions to, FireFox. This isn't true for any other database engine. When you have a DBMS every problem looks like a schema. Simon.
[sqlite] Mozilla wiki 'avoid SQLite'
On 14 Jun 2015, at 2:08pm, Jean Chevalier wrote: > Somewhat contradictory the Mozilla Foundation being a member of the SQLite > Consortium while their performance wiki prominently features a warning to > developers against using SQLite allegedly for performance reasons. Guard me > from my friends... > > http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature It's an interesting page, and I have no argument with most of it. What they're doing is not comparing SQLite and some other database engine, but comparing SQLite with a flat text file. It can be temping to write less code and use SQLite as if it's the perfect solution to every storage problem, but sometimes if you have something simple to store and search, nothing beats a plaintext file. You just have to write your search code yourself. That page is a good summary of many 'gotchas' for using those using SQLite for the first time. As long as they don't take it too seriously. Simon.
[sqlite] Mozilla wiki 'avoid SQLite'
The irony of your comment (which I entirely agree with) is that because SQLite (and similar) does so much incredibly important stuff for you, it ends up being difficult for programmers to use it, especially the lower level the programming is. They don't understand the model of how the system works, and how its meant to be used. Quite often they think they can put a value in a field, and all will be well, because that's what SQL promises. They don't think about transactions, normalization, referential integrity or anything else, because they don't need those features. But unfortunately if you don't then SQL will perform poorly (or incorrectly) because it isn't a simple system to store data. For example, I remember encountering a system where the programmers tracked a trade though various states. Each time it changed states the system would move (ie, insert then delete) the record from one table, into a nearly identical table (without a transaction). They essentially used the table to denote the state. Might make sense in an imperative model, where buffers are common, but not at all in SQL. All they're saying here is stop misusing SQLite: if a child cuts itself with a knife, take away the knife. On Sun, Jun 14, 2015 at 1:19 PM, Richard Hipp wrote: > On 6/14/15, Scott Hess wrote: > > SQLite essentially gives you a set of > > b-trees with syntactic sugar over them, > > SQL (and I speak in general terms here, not just of SQLite) provides > way more than syntactic sugar over b-trees. The syntactic sugar over > b-trees part is the *least* of the many advantages of SQL. Other more > important features include: > > (1) Transactions. Atomic updates with the option to ROLLBACK at any time. > > (2) Data abstraction. The schema concisely and accurately describes > the data format in a way that is portable across systems and across > time. Content stored in an SQL database is far, far more likely to be > accessible 25 years in the future. It is also far more likely to be > accessible today by other applications written in different > programming languages or implemented on obscure and/or experimental > hardware or operating systems. > > (3) Declarative Programming. With SQL, the programmer asks the > machine a question and lets the query planner figure out an > appropriate algorithm. A few lines of query text replace hundreds or > thousands of lines of procedural code needed to implement that query. > If performance problems are encountered, they can usually be remedied > by CREATE INDEX, and without changing a single line of code - the > query procedures devised by the query planner shift automatically. > > To be sure, many programmers end up using SQL as a glorified key-value > store. You can actually find programs that do "SELECT * FROM table > WHERE pk=?" on two or three tables and then implement a join as > procedural code in the application. Maybe this is what they were > taught. Or maybe they never really understood the concept of a join. > I dunno. But for whatever reason, you do see a lot of people treating > SQL as little more than an interface or wire protocol for talking to a > btree, and thereby bypassing 95% of the goodness of SQL. But just > because a lot of programmers do this, does not mean it is right. > > Alexander Lloyd (on the Google Spanner team) says that "NoSQL > databases ... enforce a broadly applied premature optimization on the > entire system." What Alexander means is that there are some > specialized cases where NoSQL-type solutions are required for > performance, but that those situations are the exception rather than > the rule. It is better to use the powerful abstractions provided by > SQL to build a reliable system, then go back and optimize any > performance bottlenecks on a case-by-case basis. Alexander's remarks > implicitly but clearly reference Don Knuth's assertion that premature > optimization is the root of all evil. > > Another way of thinking about the difference between SQL and NoSQL is > by analogy to C versus assembly language. There was a huge debate in > the software industry in the 1980s about whether applications should > be coded in assembly or C. Hand-coded assembly is theoretically > faster than machine-code generated by a C compiler (or at least it was > in the 80s - that point is debatable now). But in practice, programs > written in C tended to perform better than those written in assembly. > The reason is that C being higher level enabled programmers to spend > less time fiddling with bits and more time developing better > algorithms. C enables a programmer to spend more time "heads-up" > thinking about solving the users problem and less them "heads-down" > worrying about the details of the implementation. The SQL vs. NoSQL > situation is quite similar. You can, in theory, always run faster > with NoSQL storage. But in practice, SQL is so much more powerful > that applications coded using SQL tend to perform better, be more >
[sqlite] Mozilla wiki 'avoid SQLite'
On 6/14/15, Scott Hess wrote: > SQLite essentially gives you a set of > b-trees with syntactic sugar over them, SQL (and I speak in general terms here, not just of SQLite) provides way more than syntactic sugar over b-trees. The syntactic sugar over b-trees part is the *least* of the many advantages of SQL. Other more important features include: (1) Transactions. Atomic updates with the option to ROLLBACK at any time. (2) Data abstraction. The schema concisely and accurately describes the data format in a way that is portable across systems and across time. Content stored in an SQL database is far, far more likely to be accessible 25 years in the future. It is also far more likely to be accessible today by other applications written in different programming languages or implemented on obscure and/or experimental hardware or operating systems. (3) Declarative Programming. With SQL, the programmer asks the machine a question and lets the query planner figure out an appropriate algorithm. A few lines of query text replace hundreds or thousands of lines of procedural code needed to implement that query. If performance problems are encountered, they can usually be remedied by CREATE INDEX, and without changing a single line of code - the query procedures devised by the query planner shift automatically. To be sure, many programmers end up using SQL as a glorified key-value store. You can actually find programs that do "SELECT * FROM table WHERE pk=?" on two or three tables and then implement a join as procedural code in the application. Maybe this is what they were taught. Or maybe they never really understood the concept of a join. I dunno. But for whatever reason, you do see a lot of people treating SQL as little more than an interface or wire protocol for talking to a btree, and thereby bypassing 95% of the goodness of SQL. But just because a lot of programmers do this, does not mean it is right. Alexander Lloyd (on the Google Spanner team) says that "NoSQL databases ... enforce a broadly applied premature optimization on the entire system." What Alexander means is that there are some specialized cases where NoSQL-type solutions are required for performance, but that those situations are the exception rather than the rule. It is better to use the powerful abstractions provided by SQL to build a reliable system, then go back and optimize any performance bottlenecks on a case-by-case basis. Alexander's remarks implicitly but clearly reference Don Knuth's assertion that premature optimization is the root of all evil. Another way of thinking about the difference between SQL and NoSQL is by analogy to C versus assembly language. There was a huge debate in the software industry in the 1980s about whether applications should be coded in assembly or C. Hand-coded assembly is theoretically faster than machine-code generated by a C compiler (or at least it was in the 80s - that point is debatable now). But in practice, programs written in C tended to perform better than those written in assembly. The reason is that C being higher level enabled programmers to spend less time fiddling with bits and more time developing better algorithms. C enables a programmer to spend more time "heads-up" thinking about solving the users problem and less them "heads-down" worrying about the details of the implementation. The SQL vs. NoSQL situation is quite similar. You can, in theory, always run faster with NoSQL storage. But in practice, SQL is so much more powerful that applications coded using SQL tend to perform better, be more reliable, and have more and useful features than those written using NoSQL or ad hoc storage solutions. One clear example of an application suffering from a lack of SQL is Git. Git uses a NoSQL ad hoc key/value data store - files in the .git directory. This is fast (they say). But it also causes serious problems. For example, finding the children of a commit involves a full-table scan of the Git log. This turns out to be so difficult and slow that nobody implements it. On GitHub (and on every other Git interface I've seen) you will find links to the parents of a check-in but not its children. And so when looking at the history of a Git project, it is impossible to find out what happened next. It also leads to the vexing problem known as "detached head state". If the Git log had been implemented as an SQL database (SQLite would have been a good choice for this, I think, but there are other options) then a detached head becomes impossible - that scourge of so many hapless Git users simple would have never existed. And finding (for example) the first 30 descendants of a check-in would become a simple query something like this: WITH RECURSIVE dx(h,m) AS (SELECT child, mtime FROM gitlog WHERE child=$commithash UNION SELECT child, mtime FROM gitlog, dx WHERE parent=h ORDER BY 2) SELECT h,
[sqlite] Mozilla wiki 'avoid SQLite'
On 6/14/2015 3:00 PM, Simon Slavin wrote: > The result is that that higher level the language you write in, the better. I disagree. The use of languages higher than C result in slow bloated code. A language that calls a language that calls a language. Simple programs become multi-megabyte resource hogs. I agree that C compilers are able to optimize assembler code to a level that hand-coded assembler probably could not achieve. The problem is that higher level languages are not compiling to assembler, but compiling to the language below it. Scott Doctor scott at scottdoctor.com --
[sqlite] Mozilla wiki 'avoid SQLite'
Somewhat contradictory the Mozilla Foundation being a member of the SQLite Consortium while their performance wiki prominently features a warning to developers against using SQLite allegedly for performance reasons. Guard me from my friends... http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature
[sqlite] Mozilla wiki 'avoid SQLite'
On Sun, 14 Jun 2015, Richard Hipp wrote: > SQL (and I speak in general terms here, not just of SQLite) provides way > more than syntactic sugar over b-trees. The syntactic sugar over b-trees > part is the *least* of the many advantages of SQL. Other more important > features include: Very well written, Richard. > (3) Declarative Programming. With SQL, the programmer asks the machine a > question and lets the query planner figure out an appropriate algorithm. A > few lines of query text replace hundreds or thousands of lines of > procedural code needed to implement that query. If performance problems > are encountered, they can usually be remedied by CREATE INDEX, and without > changing a single line of code - the query procedures devised by the query > planner shift automatically. Another stumbling block for some application developers is not recognizing that SQL works in sets unlike earlier database formats that stored fields in records. That's why a SELECT returns an entire set of table rows meeting the selection criteria and the row order is not guaranteed. Rich
[sqlite] Mozilla wiki 'avoid SQLite'
Most of that looks to be more like common sense things rather than SQLite specific, so why they're calling out SQLite, I've no idea. Also, this doc was last modified more than a year ago. Stuff has changed both within FF as well as SQlite since then. I've just deleted the blog I was going to write here, since I know I'm pissing people off with my long thoughts (And I'm tired), so yeah... I'll cut it to a paragraph. *Old data, common sense to clean up sometime during the current apps life cycle, or something completely different spawned after the primary apps life cycle with low priority. *Proper knowledge of how SQLite handles indexing of data is required (Looked at that bug report linked). *Fragmentation happens, within and outside databases. *Database page sizes should be close to your "common" target platforms disk cluster size to reduce disk IO. *And yeah... I LOL'd at "JSON files or log files will show better I/O patterns almost every time, especially if they're compressed and read/written in entirety each time", specifically at JSON, and the concept that substituting disk IO for CPU time to decompress then ANALYZE the full content of that data, then delete previous data to recompress data is any better, ESPECIALLY when trickling data into a compressed JSON log file. Not a fan of that JSON fanboi tech at all. On Sun, Jun 14, 2015 at 9:08 AM, Jean Chevalier wrote: > Somewhat contradictory the Mozilla Foundation being a member of the SQLite > Consortium while their performance wiki prominently features a warning to > developers against using SQLite allegedly for performance reasons. Guard me > from my friends... > > > http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users >
[sqlite] Mozilla wiki 'avoid SQLite'
Jean Chevalier wrote: > Somewhat contradictory the Mozilla Foundation being a member of the > SQLite Consortium while their performance wiki prominently features a > warning to developers against using SQLite allegedly for performance > reasons. Guard me from my friends... > > http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature That page describes situations where SQLite doesn't give best available performance, reduce complexity, or provide needed features, and then goes on to justify those conclusions (including recognizing how SQLite isn't unique in these respects: "This isn't an indictment of SQLite itself -- any other relational embedded DB would pose the same challenges."). I'm not sure what point you were raising in your post, so I'll have to guess. Your summary suggests that you expected SQLite Consortium members will endorse SQLite even in situations where SQLite isn't a good fit due to preferring other tradeoffs. Are SQLite Consortium members somehow obliged to endorse SQLite for tasks even when other approaches present more desirable tradeoffs? If so, where is this obligation published?
[sqlite] Mozilla wiki 'avoid SQLite'
For various reasons I've ended up as the SQLite rep w/in Chromium, and I bookmarked that page awhile back to periodically revisit. People often seem to believe that SQLite magically solves tons of problems with their persistence layer, without realizing that many of their assumptions are based on things like having a DBA to take care of things behind the scenes, or having a distinct database server process probably running on pretty beefy hardware. Also, to be honest, I find that developers often don't really _think_ about what they're doing. SQLite essentially gives you a set of b-trees with syntactic sugar over them, so if you cannot adequately describe how your schema works in terms of raw b-trees, then SQLite isn't going to be able to magically make things awesome behind your back. They also don't think about the evolution of their schema over time. So you see developers copy/paste some other code and make it worse, and then never float things by an expert to see if they are making poor decisions, and three years later someone has to disentangle the horror. Also, heaven defend me from people writing ad-hoc array-based query systems that automatically generate bad SQL. My experience with ORM systems over the years has convinced me that if you spin up an entire dedicated team to work really hard on that problem alone for years, it will still be quite unsatisfying in the end. One developer making a bunch of stuff up and then abandoning it is just a recipe for later pain. There are many places where I think developers would have been better served with a shared nosql-type store (like leveldb) mapping keys to something a decent serialization system (like protocol buffers), and which would have some well-defined and automated procedures for handling things like transaction scheduling and recovery from filesystem corruption and other errors. It wouldn't be as good as SQLite can be, but it wouldn't be as bad as the way many people use SQLite in practice. [To be fair to SQLite, much of my position on this isn't much different than my position on MySQL in a past position. SQL is a challenge for many developers.] -scott On Sun, Jun 14, 2015 at 6:08 AM, Jean Chevalier wrote: > Somewhat contradictory the Mozilla Foundation being a member of the SQLite > Consortium while their performance wiki prominently features a warning to > developers against using SQLite allegedly for performance reasons. Guard me > from my friends... > > http://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature > ___ > sqlite-users mailing list > sqlite-users at mailinglists.sqlite.org > http://mailinglists.sqlite.org/cgi-bin/mailman/listinfo/sqlite-users
[sqlite] Mozilla wiki 'avoid SQLite'
On Jun 14, 2015 9:43 AM, "Stephen Chrzanowski" wrote: > > Most of that looks to be more like common sense things rather than SQLite > specific, so why they're calling out SQLite, I've no idea. Also, this doc > was last modified more than a year ago. Stuff has changed both within FF > as well as SQlite since then. They call out SQLite because that is the engine they embed in their infrastructure. They aren't worried about people in their code base trying to use MySQL, PgSQL, SQL Server, or Oracle. They're worried about their developers using SQLite when some other format would be more appropriate for some fuzzy value of "appropriate". One shouldn't use a hammer on screws or a screwdriver on nails. Use the right tool for the job. Sometimes FILE* based functionality is the right tool. Sometimes not. It all comes down to the fuzzy concept of appropriate.
