Re: SQLite-D alpha is here [did it mention it works at CTFE]
Support for reading Index-Trees and there(WITHOUT ROWID) tables, has landed in master! Now the real fun can start. CTFE Query translation and optimization.
Re: SQLite-D alpha is here [did it mention it works at CTFE]
On Friday, 27 May 2016 at 07:08:30 UTC, Stefan Koch wrote: On Friday, 27 May 2016 at 05:12:33 UTC, Stefan Koch wrote: There is a locking mechanism in SQLite-proper. However SQLite-D currently makes no attempt in looking for the lock-page. Opps I was wrong :) The Lock-page has nothing todo with threading :) Maybe the thread in general got lost :) SQLite-D works at compile-time.
Re: SQLite-D alpha is here
On Friday, 27 May 2016 at 05:12:33 UTC, Stefan Koch wrote: There is a locking mechanism in SQLite-proper. However SQLite-D currently makes no attempt in looking for the lock-page. Opps I was wrong :) The Lock-page has nothing todo with threading :)
Re: SQLite-D alpha is here
On Friday, 27 May 2016 at 05:03:43 UTC, xenon325 wrote: For example, I have one thread with traditional (slow) SQLite client, which seldom updates data. And another thread which reads data with sqlite-d. Will not program crash or read trash/inconsistent data ? sqlite-d provides no safety whatsoever. You will the database read in whatever state the operation system see the underlying file. It is unlikely that it will crash. But of course it may read stale or incomplete data. There is a locking mechanism in SQLite-proper. However SQLite-D currently makes no attempt in looking for the lock-page. As a side note: I would like to make clear that SQLite is very fast for what it does. It is among the fastest of SQL-databases, and has a high quality of implementation.
Re: SQLite-D alpha is here
On Saturday, 27 February 2016 at 16:08:21 UTC, Stefan Koch wrote: Hello, I am happy to announce the official alpha version of sqlite-d! sqlite-d is a reader for the SQLite File Format 3. On Thursday, 26 May 2016 at 19:11:50 UTC, Stefan Koch wrote: So, all that sqlite-d does is reading the sqlite-db files. However the proper sqlite does much more: It implements a whole ByteCode-Compiler and Interpreter to be able to precompile and execute SQL-statements. Thus even for simple querys you have a significant overhead. If you want to use sqlite-d you basically write all your queries as D code which can be directly executed, rather then having to be compiled from SQL to ByteCode. How does that play with transactions and ACID in general ? For example, I have one thread with traditional (slow) SQLite client, which seldom updates data. And another thread which reads data with sqlite-d. Will not program crash or read trash/inconsistent data ?
Re: SQLite-D alpha is here
On Thursday, 26 May 2016 at 19:35:22 UTC, Suliman wrote: Oh! Look like for all time I misunderstood the purpose of your project. Do you mean that your tool is created not for being SQL compatible driver, but make possible to use D code for iteration with DB? Yes and no, currently it does only provide primitives to iterate over sqlite-tables. And originally it was only intended for this purpose. However per-compiling SQL at compile-time is a great way for D's features to shine. And does it's mean that is not possible to write SQL-query with this driver? I want to add the capability to precompile and execute SQL, however that is still a long way away.
Re: SQLite-D alpha is here
On Thursday, 26 May 2016 at 19:11:50 UTC, Stefan Koch wrote: On Thursday, 26 May 2016 at 14:45:58 UTC, Suliman wrote: Could you explain more details? What do you mean by indirection work with data? Sure, I can explain. So, all that sqlite-d does is reading the sqlite-db files. However the proper sqlite does much more: It implements a whole ByteCode-Compiler and Interpreter to be able to precompile and execute SQL-statements. Thus even for simple querys you have a significant overhead. If you want to use sqlite-d you basically write all your queries as D code which can be directly executed, rather then having to be compiled from SQL to ByteCode. Oh! Look like for all time I misunderstood the purpose of your project. Do you mean that your tool is created not for being SQL compatible driver, but make possible to use D code for iteration with DB? And does it's mean that is not possible to write SQL-query with this driver?
Re: SQLite-D alpha is here
On Thursday, 26 May 2016 at 14:45:58 UTC, Suliman wrote: Could you explain more details? What do you mean by indirection work with data? Sure, I can explain. So, all that sqlite-d does is reading the sqlite-db files. However the proper sqlite does much more: It implements a whole ByteCode-Compiler and Interpreter to be able to precompile and execute SQL-statements. Thus even for simple querys you have a significant overhead. If you want to use sqlite-d you basically write all your queries as D code which can be directly executed, rather then having to be compiled from SQL to ByteCode.
Re: SQLite-D alpha is here
On Wednesday, 25 May 2016 at 14:10:41 UTC, Stefan Koch wrote: On Wednesday, 25 May 2016 at 06:52:22 UTC, Suliman wrote: Even faster then official version? What about futures, would it possible to make it's 100% compatibility with C version? Not really. The reason why it is faster is because there is no indirection in working with the data. If I had full SQL support like there is in the C version, and you take that route rather then the direct access. It will be as slow as the C version, and probably slower. Could you explain more details? What do you mean by indirection work with data?
Re: SQLite-D alpha is here
On Wednesday, 25 May 2016 at 03:48:48 UTC, Stefan Koch wrote: This project is currently on the back burner. However in the next days there will be another significant performance improvement :) A 20% performance improvement has landed in master! It is possible that there are more places in the code-base where unrolling can buy you performance. Never forget. PERFMATTERS!
Re: SQLite-D alpha is here
On Wednesday, 25 May 2016 at 03:48:48 UTC, Stefan Koch wrote: Well not much has changed since I made this announcement. By fixing a really jarring bug I had a slight performance regression. But this is still the fastest SQLite reader I know of. This project is currently on the back burner. However in the next days there will be another significant performance improvement :) You might consider writing up some kind of comparison.
Re: SQLite-D alpha is here
On Wednesday, 25 May 2016 at 06:52:22 UTC, Suliman wrote: Even faster then official version? What about futures, would it possible to make it's 100% compatibility with C version? Not really. The reason why it is faster is because there is no indirection in working with the data. If I had full SQL support like there is in the C version, and you take that route rather then the direct access. It will be as slow as the C version, and probably slower.
Re: SQLite-D alpha is here
On Wednesday, 25 May 2016 at 03:48:48 UTC, Stefan Koch wrote: On Saturday, 27 February 2016 at 16:08:21 UTC, Stefan Koch wrote: Hello, I am happy to announce the official alpha version of sqlite-d! sqlite-d is a reader for the SQLite File Format 3. In the future I will implement a SQL-like API on top of it. Top-notch performance is one of the explicit goals of this project! please note that currently only the ctfe branch is populated. I welcome contributions! Repo-URL : https://github.com/UplinkCoder/sqlite-d Well not much has changed since I made this announcement. By fixing a really jarring bug I had a slight performance regression. But this is still the fastest SQLite reader I know of. This project is currently on the back burner. However in the next days there will be another significant performance improvement :) Even faster then official version? What about futures, would it possible to make it's 100% compatibility with C version?
Re: SQLite-D alpha is here
On Saturday, 27 February 2016 at 16:08:21 UTC, Stefan Koch wrote: Hello, I am happy to announce the official alpha version of sqlite-d! sqlite-d is a reader for the SQLite File Format 3. In the future I will implement a SQL-like API on top of it. Top-notch performance is one of the explicit goals of this project! please note that currently only the ctfe branch is populated. I welcome contributions! Repo-URL : https://github.com/UplinkCoder/sqlite-d Well not much has changed since I made this announcement. By fixing a really jarring bug I had a slight performance regression. But this is still the fastest SQLite reader I know of. This project is currently on the back burner. However in the next days there will be another significant performance improvement :)
Re: SQLite-D alpha is here
On Sunday, 28 February 2016 at 12:14:14 UTC, Stefan Koch wrote: Update I just found another case that cannot be handled properly. It just happens with insanely huge databases. That bug is fixed now!
Re: SQLite-D alpha is here
I made a huge performance improvement sqlite-d is now 6-8 times faster then on the day were it wad able to read the first payloads. 2. I am heavily working on write-support. 3.Sqlite-d will then implement the allocator interface! (although I am flexible on that should it turn out to be a bad idea. I wish you all a happy day!
Re: SQLite-D alpha is here
On Saturday, 27 February 2016 at 16:08:21 UTC, Stefan Koch wrote: Hello, I am happy to announce the official alpha version of sqlite-d! sqlite-d is a reader for the SQLite File Format 3. In the future I will implement a SQL-like API on top of it. Top-notch performance is one of the explicit goals of this project! please note that currently only the ctfe branch is populated. I welcome contributions! Repo-URL : https://github.com/UplinkCoder/sqlite-d Update I just found another case that cannot be handled properly. It just happens with insanely huge databases. Also I commited my test-data now... I wonder why nobody said anything about the app failing on startup ...
Re: SQLite-D alpha is here
On Saturday, 27 February 2016 at 17:03:17 UTC, Adam D. Ruppe wrote: On Saturday, 27 February 2016 at 16:54:49 UTC, Suliman wrote: Why? etc.c is for the C interface. This is not the C interface. Besides, the original code will surely be ahead of features and compatibility for a long time anyway. The duration of that time depends heavily on the amount of contributions :) But the short term goal is not to reinvent SQLite rather to present a file-format compatible alternative. The big advantage of sqlite-d is that you can write your logic in D without having to worry about generating SQL or converting your data into strings. However in the long-term I plan to provide SQL support if there is sufficient demand for it. Right now I am still experimenting with different approaches to make sure the API will not compromise performance. Also I pushed ctfe to the master-branch now. the repo no longer looks empty.
Re: SQLite-D alpha is here
On Saturday, 27 February 2016 at 16:54:49 UTC, Suliman wrote: Why? etc.c is for the C interface. This is not the C interface. Besides, the original code will surely be ahead of features and compatibility for a long time anyway.
Re: SQLite-D alpha is here
On Saturday, 27 February 2016 at 16:35:05 UTC, ketmar wrote: no. Why?
Re: SQLite-D alpha is here
no.
