On Wed, 16 Nov 2011 14:53:08 +0200 Juha Manninen <[email protected]> wrote:
> Hi > > Mattias, you mentioned some time ago that you were busy with a new web > server project that uses XML database which will become an open source > project. > Could you please explain briefly what the "XML database" actually means. It's a lightweight database for XML files. That means you give it a list of directories and it will load/parse all xml files including subdirectories into memory for fast access. You can query via simple HTTP requests and get xml. So you are not limited to FPC, you can use any language. There are several types of query, including some with XPath expressions. At the moment it supports only a few XPath features, but they are already quite powerful. When a file was edited/changed you can trigger a rescan or you can rescan in interval. A rescan on a thousand directories to search for modified files usually takes 10-100ms and is done in separate threads to not disturb the queries. It parses about 5-20MB per second per core and automatically uses all cores. It is currently used by two projects, that were migrated from eXist DB. One is running on an OS X server, the other on a Linux server. They have thousands of directories, more than 150MB of XML files, which refer to others in various ways. Because it only needs xml files, the normal ugly database tasks like setting up data structures, uploading new data, editing, cleaning up, renaming, backup, restore and upgrade are a no brainer. Even running it on a cluster is simple - you only need a share. For example one project uses it like this: The users use their favorite tool to edit the xml files and after they committed to the svn server they switch to their browser and instantly see their changes on the website. The xml database is only one part of the project(s). The queries are done via some instantfpc scripts. > This is interesting because often the DB queries are the slowest part of a > (web-)application. Yes, that was one of the reasons to switch from eXist DB. The amazing thing is, that for the current projects it was sufficient to implement only brute force searches, without any index and they still are more than five times faster and do more complex stuff. To be fair: eXist DB is quite fast. With more man power it would have been possible to speed up the eXist implementation. The other reasons to implement a new xml database were debugging and usability. The database is a single executable, so each developer can simply run it locally and test every step with all the usual debugger tools. Usability: The users only have to handle files and directories. Well, I must admit, for some users even that is too much, but usually they quickly get used to it. It is far easier than the former work flow. > I was discussing about PHP's performance and I was told it is not important > compared to DB query delays. > Unfortunately it is true, even if table indexes are done right. > You can save milliseconds by optimizing your code or using a compiled > language but who cares if the DB query takes one second. > Sites that have an extremely heavy load, like FaceBook, make an exception. > Then PHP starts to use too much CPU and memory resources. > > I ask on this general list because it may be of interest to others, too. The project is not yet published. I have to clean up a few things and write a basic documentation. Then it will be published. License will be free for commercial use. Mattias -- _______________________________________________ Lazarus mailing list [email protected] http://lists.lazarus.freepascal.org/mailman/listinfo/lazarus
