Re: [Dovecot] Berkeley DB with Dovecot
On Mon, May 21, 2012 at 02:37:28PM +0200, Sven Hartge wrote: Jerry je...@seibercom.net wrote: On Mon, 21 May 2012 06:14:10 -0400 Charles Marcus articulated: Out of curiousity... How is the performance of SQLite? I'm assuming it is only recommended for servers that are not under heavy load... SQLite support in Postfix is fairly young. I'm not sure about Dovecot, but again I doubt it has been around as long as real RDB systems. It hasn't existed as long as they have! 2000 May 29 was the initial Alpha release. In contrast, MySQL appeared in 1995, and PostgreSQL evolved in 1996 from a much older project. SQLite.org has a very old article where someone did some read and write benchmarking against MySQL and PostgreSQL. In those tests it did very well. This is what I'd expect, since there is nothing between the application and the database. The SQLite file will be cached in system RAM if it is frequently accessed, which in a moderate mail server, it certainly would be. I don't have a busy system, but I expect it could handle anything one might throw at it. What are the main advantages/disadvantages of using SQLite over MySQL? At this point there is no easy frontend for SQLite-managed mail servers. If you want to give a non-technical user the ability to manage accounts, at this point, you'd probably have to write a GUI interface. (That should not be a problem for a competent programmer in just about any language.) Being younger code, it is less well tested. If you recall, I stumbled upon a potentially serious bug in Postfix, fixed in 2.8.9 and 2.9.0. Poorly managed file permissions could have led to SQL injection attacks. (But Postfix does complain about it if the logs is not owned by and only writable by root.) SQLite.org has a good when to use SQLite page: http://sqlite.org/whentouse.html I found numerous links for just that sort of information on Google Bing. These two seem rather informative. http://stackoverflow.com/questions/2824135/how-fast-is-berkeley-db-sql-compared-to-sqlite http://www.oracle.com/technetwork/database/berkeleydb/learnmore/bdbvssqlite-wp-186779.pdf Hmm. Those documenets only talk about heavy writing to the database which is not involved in the Dovecot scenario discussed here, where the database is used as a data storage for the configuration which is mostly read. Note that the same is true of BDB. I wouldn't expect SQLite to beat BDB on raw read speed, but I'd expect it to show respectably, as did the Oracle author (as he stated in the stackoverflow page.) SQLite might vary widely depending on the schema and SQL queries. BDB, OTOH, is not going to vary in that regard: it's only a Key-to- Value mapping. In comparing SQLite to BDB, I did optimize my own setup to reduce some of the multiple queries Postfix does. If a lookup of user@domain has no result, my queries also check for domain, and that value would be returned. So it's possible that 3 simple BDB lookups are going to be beaten by 1-2 complex SQL queries in SQLite. So the question, how fast SQLite is during read operations compared to BDB is still unanswered. Indeed it is not. We need someone to do the testing. :) -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if /dev/rob0 is in the Subject:
Re: [Dovecot] Berkeley DB with Dovecot
:2012-05-22T18:20:/dev/rob0: What are the main advantages/disadvantages of using SQLite over MySQL? Well the main one... You don't have a full blown RDBMS running which in my opinion is a waste of resources. Anyway I also have a SQLite based Dovecot and Exim setup. In case you care to take a look at my setup(note the php part is nowhere near done/complete or even tested): http://repo.or.cz/w/virtuo.git The shell frontend though works. -- Andraž 'ruskie' Levstik Source Mage GNU/Linux Games/Xorg grimoire guru Re-Alpine Coordinator http://sourceforge.net/projects/re-alpine/ Geek/Hacker/Tinker Ryle hira.
Re: [Dovecot] Berkeley DB with Dovecot
On 2012-05-20 1:48 PM, Luuk@dovecot dove...@vosslamber.nl wrote: SQlite is only 61Kb, no config is needed If you start using MySQL, and the MySQL-database is changed because of some upgrades to the system that currenly uses it, it might have an impact on your mail-system For that 2 cents, i would install SQlite! Out of curiousity... How is the performance of SQLite? I'm assuming it is only recommended for servers that are not under heavy load... What are the main advantages/disadvantages of using SQLite over MySQL? Thanks, Charles
Re: [Dovecot] Berkeley DB with Dovecot
On Mon, 21 May 2012 06:14:10 -0400 Charles Marcus articulated: On 2012-05-20 1:48 PM, Luuk@dovecot dove...@vosslamber.nl wrote: SQlite is only 61Kb, no config is needed If you start using MySQL, and the MySQL-database is changed because of some upgrades to the system that currenly uses it, it might have an impact on your mail-system For that 2 cents, i would install SQlite! Out of curiousity... How is the performance of SQLite? I'm assuming it is only recommended for servers that are not under heavy load... What are the main advantages/disadvantages of using SQLite over MySQL? I found numerous links for just that sort of information on Google Bing. These two seem rather informative. http://stackoverflow.com/questions/2824135/how-fast-is-berkeley-db-sql-compared-to-sqlite http://www.oracle.com/technetwork/database/berkeleydb/learnmore/bdbvssqlite-wp-186779.pdf -- Jerry ♔ Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __
Re: [Dovecot] Berkeley DB with Dovecot
Jerry je...@seibercom.net wrote: On Mon, 21 May 2012 06:14:10 -0400 Charles Marcus articulated: On 2012-05-20 1:48 PM, Luuk@dovecot dove...@vosslamber.nl wrote: SQlite is only 61Kb, no config is needed If you start using MySQL, and the MySQL-database is changed because of some upgrades to the system that currenly uses it, it might have an impact on your mail-system For that 2 cents, i would install SQlite! Out of curiousity... How is the performance of SQLite? I'm assuming it is only recommended for servers that are not under heavy load... What are the main advantages/disadvantages of using SQLite over MySQL? I found numerous links for just that sort of information on Google Bing. These two seem rather informative. http://stackoverflow.com/questions/2824135/how-fast-is-berkeley-db-sql-compared-to-sqlite http://www.oracle.com/technetwork/database/berkeleydb/learnmore/bdbvssqlite-wp-186779.pdf Hmm. Those documenets only talk about heavy writing to the database which is not involved in the Dovecot scenario discussed here, where the database is used as a data storage for the configuration which is mostly read. So the question, how fast SQLite is during read operations compared to BDB is still unanswered. Grüße, Sven. -- Sigmentation fault. Core dumped.
Re: [Dovecot] Berkeley DB with Dovecot
On 5/20/2012 6:19 AM, Jerry wrote: I have a friend who is preparing to set up a small Postfix/Dovecot mail system. There are only approximately 25 users. He wants to use Berkeley DB in a similar fashion to the way Postfix does. I told him I do not believe Dovecot supports that. I could not find any documentation relating to it. 1) Does Dovecot support Berkeley DB? No: http://wiki.dovecot.org/AuthDatabase These databases can be used as both password databases and user databases: Passwd: System users (NSS, /etc/passwd, or similiar) Passwd-file: /etc/passwd-like file in specified location LDAP: Lightweight Directory Access Protocol SQL: SQL database (PostgreSQL, MySQL, SQLite) VPopMail: External software used to handle virtual domains 2) Are their any plans to incorporate that feature into Dovecot? It [Timo answers here] seems like it could potentially be a very useful feature. I have used it myself when a full blown MySQL configuration seemed like over-kill on other small projects. SQLite is the appropriate solution for this scenario. -- Stan
Re: [Dovecot] Berkeley DB with Dovecot
On 5/20/2012 1:19 PM, Jerry wrote: I have a friend who is preparing to set up a small Postfix/Dovecot mail system. There are only approximately 25 users. He wants to use Berkeley DB in a similar fashion to the way Postfix does. I told him I do not believe Dovecot supports that. I could not find any documentation relating to it. 1) Does Dovecot support Berkeley DB? 2) Are their any plans to incorporate that feature into Dovecot? It seems like it could potentially be a very useful feature. I have used it myself when a full blown MySQL configuration seemed like over-kill on other small projects. http://www.mail-archive.com/dovecot@dovecot.org/msg22777.html I don't think anything has changed since then. Regards, Stephan
Re: [Dovecot] Berkeley DB with Dovecot
On Sun, 20 May 2012 06:35:46 -0500 Stan Hoeppner articulated: On 5/20/2012 6:19 AM, Jerry wrote: I have a friend who is preparing to set up a small Postfix/Dovecot mail system. There are only approximately 25 users. He wants to use Berkeley DB in a similar fashion to the way Postfix does. I told him I do not believe Dovecot supports that. I could not find any documentation relating to it. 1) Does Dovecot support Berkeley DB? No: http://wiki.dovecot.org/AuthDatabase I have all ready read that. 2) Are their any plans to incorporate that feature into Dovecot? It [Timo answers here] Are you ordering Timo to answer or giving him the option? In any case, I am sure he will appreciate you leaving him space to enter his comments. seems like it could potentially be a very useful feature. I have used it myself when a full blown MySQL configuration seemed like over-kill on other small projects. SQLite is the appropriate solution for this scenario. Requires loading another application when it is not necessary. Most scenarios I have come across have Berkeley DB all ready installed. In fact, it is an extremely useful tool for checking that things work as expected under Postfix prior to migrating to a full blown SQL database. By the way, I was informed that MySQL is installed on the system in question; however, my friend was advised against using it at the present time. Details NOT available. -- Jerry ♔ Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __
Re: [Dovecot] Berkeley DB with Dovecot
On Sun, 20 May 2012 13:58:41 +0200 Stephan Bosch articulated: On 5/20/2012 1:19 PM, Jerry wrote: 1) Does Dovecot support Berkeley DB? 2) Are their any plans to incorporate that feature into Dovecot? It seems like it could potentially be a very useful feature. I have used it myself when a full blown MySQL configuration seemed like over-kill on other small projects. http://www.mail-archive.com/dovecot@dovecot.org/msg22777.html I don't think anything has changed since then. Thanks Stephan, that is the sort of information that I was looking for. -- Jerry ♔ Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __ Obviously the only rational solution to your problem is suicide.
Re: [Dovecot] Berkeley DB with Dovecot
On 2012-05-20 8:11 AM, Jerry je...@seibercom.net wrote: On Sun, 20 May 2012 06:35:46 -0500 Stan Hoeppner articulated: On 5/20/2012 6:19 AM, Jerry wrote: 2) Are their any plans to incorporate that feature into Dovecot? It [Timo answers here] Are you ordering Timo to answer or giving him the option? No... he meant that what followed (which I have repeated below for your edification) was Timo's answer to this very question some time ago... seems like it could potentially be a very useful feature. I have used it myself when a full blown MySQL configuration seemed like over-kill on other small projects. SQLite is the appropriate solution for this scenario. So, Timo said that the appropriate solution for this scenario (not wanting to use a full blown mysql/pgsql environment) was to use SQLite... -- Best regards, Charles
Re: [Dovecot] Berkeley DB with Dovecot
On Sun, May 20, 2012 at 08:11:33AM -0400, Jerry wrote: On Sun, 20 May 2012 06:35:46 -0500 Stan Hoeppner articulated: On 5/20/2012 6:19 AM, Jerry wrote: seems like it could potentially be a very useful feature. I have used it myself when a full blown MySQL configuration seemed like over-kill on other small projects. SQLite is the appropriate solution for this scenario. Requires loading another application when it is not necessary. What application in particular? SQLite is a file format, not a daemon. Both Dovecot and Postfix support it well (in the latter case, use at least 2.8.9 or 2.9 due to a bug.) sqlite3(1) is the console client for SQLite. You can use that or any other SQLite client to maintain your database file. My own sordid SQLite mail server howto is linked at the .sig site. -- http://rob0.nodns4.us/ -- system administration and consulting Offlist GMX mail is seen only if /dev/rob0 is in the Subject:
Re: [Dovecot] Berkeley DB with Dovecot
On Sun, 20 May 2012 10:13:45 -0500 /dev/rob0 articulated: On Sun, May 20, 2012 at 08:11:33AM -0400, Jerry wrote: On Sun, 20 May 2012 06:35:46 -0500 Stan Hoeppner articulated: On 5/20/2012 6:19 AM, Jerry wrote: seems like it could potentially be a very useful feature. I have used it myself when a full blown MySQL configuration seemed like over-kill on other small projects. SQLite is the appropriate solution for this scenario. Requires loading another application when it is not necessary. What application in particular? SQLite is a file format, not a daemon. Both Dovecot and Postfix support it well (in the latter case, use at least 2.8.9 or 2.9 due to a bug.) sqlite3(1) is the console client for SQLite. You can use that or any other SQLite client to maintain your database file. My own sordid SQLite mail server howto is linked at the .sig site. What I meant was that SQLite would have to be installed on the system. As far as I know, it is not; although MySQL is. I just talked to my friend and he told me the problem was that the owner of the company was/is afraid of damaging the databases presently in use. We are going to assure him that no such thing will happen and hopefully put his mind at ease. At present all of their mail is handled by a third party and they want to now handle it all in house. I still think that supporting Berkeley DB is a worthwhile goal; however that is my own 2¢. -- Jerry ♔ Disclaimer: off-list followups get on-list replies or get ignored. Please do not ignore the Reply-To header. __
Re: [Dovecot] Berkeley DB with Dovecot
Am 20.05.2012 19:00, schrieb Jerry: What I meant was that SQLite would have to be installed on the system. As far as I know, it is not; although MySQL is. I just talked to my friend and he told me the problem was that the owner of the company was/is afraid of damaging the databases presently in use. We are going to assure him that no such thing will happen and hopefully put his mind at ease. why in the world should existing databases get damaged by another one? it is the main job of a db-server to offer as many databases/users/tables as needed and since if you are using MyISAM each database have it's own folder which you can even copy offline to any other server how should any damage be technically possible only because one more db? stats of one of our mysql-servers: * databases: 195 * tables: 4652 * records: 329049 signature.asc Description: OpenPGP digital signature
Re: [Dovecot] Berkeley DB with Dovecot
On 20-05-2012 19:00, Jerry wrote: On Sun, 20 May 2012 10:13:45 -0500 /dev/rob0 articulated: On Sun, May 20, 2012 at 08:11:33AM -0400, Jerry wrote: On Sun, 20 May 2012 06:35:46 -0500 Stan Hoeppner articulated: On 5/20/2012 6:19 AM, Jerry wrote: seems like it could potentially be a very useful feature. I have used it myself when a full blown MySQL configuration seemed like over-kill on other small projects. SQLite is the appropriate solution for this scenario. Requires loading another application when it is not necessary. What application in particular? SQLite is a file format, not a daemon. Both Dovecot and Postfix support it well (in the latter case, use at least 2.8.9 or 2.9 due to a bug.) sqlite3(1) is the console client for SQLite. You can use that or any other SQLite client to maintain your database file. My own sordid SQLite mail server howto is linked at the .sig site. What I meant was that SQLite would have to be installed on the system. As far as I know, it is not; although MySQL is. I just talked to my friend and he told me the problem was that the owner of the company was/is afraid of damaging the databases presently in use. We are going to assure him that no such thing will happen and hopefully put his mind at ease. At present all of their mail is handled by a third party and they want to now handle it all in house. I still think that supporting Berkeley DB is a worthwhile goal; however that is my own 2¢. SQlite is only 61Kb, no config is needed If you start using MySQL, and the MySQL-database is changed because of some upgrades to the system that currenly uses it, it might have an impact on your mail-system For that 2 cents, i would install SQlite! # zypper info sqlite3 Information for package sqlite3: Repository: openSUSE-12.1-Oss Name: sqlite3 Version: 3.7.8-1.1.2 Arch: x86_64 Vendor: openSUSE Installed: Yes Status: up-to-date Installed Size: 61.0 KiB Summary: Embeddable SQL Database Engine Description: SQLite is a C library that implements an embeddable SQL database engine. Programs that link with the SQLite library can have SQL database access without running a separate RDBMS process. SQLite is not a client library used to connect to a big database server. SQLite is a server and the SQLite library reads and writes directly to and from the database files on disk. SQLite can be used via the sqlite command line tool or via any application that supports the Qt database plug-ins.
