Re: [Dovecot] highly available userdb

2013-11-18 Thread Jan-Frode Myklebust
On Wed, Nov 13, 2013 at 01:52:09PM +1000, Nick Edwards wrote:
 On 11/12/13, Jan-Frode Myklebust janfr...@tanso.net wrote:
  My installation is only serving 1/10 of your size, but long time ago we
  migrated off mysql for userdatabase, and over to LDAP. The MySQL data
  source had issues (not dovecot related), and didn't seem like the right
  tool for the job.
 
 
 A database is a database, a master is a master, and a slave is a slave

And some databases are better for some tasks than others. F.ex. LDAP
gives dovecot free failover between servers. Handled in the ldap
libraries. One could argue that you should be complaining to the MySQL
developers about supporting server failover in the client library, and
not to Dovecot.

 our mysql has never had problem, not a single one, its why I'm so annoyed
 dovecot is talking to master when it doesn't need to



  -jf


Re: [Dovecot] highly available userdb

2013-11-12 Thread Nick Edwards
On 11/12/13, Jan-Frode Myklebust janfr...@tanso.net wrote:
 My installation is only serving 1/10 of your size, but long time ago we
 migrated off mysql for userdatabase, and over to LDAP. The MySQL data
 source had issues (not dovecot related), and didn't seem like the right
 tool for the job.


A database is a database, a master is a master, and a slave is a slave
our mysql has never had problem, not a single one, its why I'm so annoyed
dovecot is talking to master when it doesn't need to


Re: [Dovecot] highly available userdb (Was: Re Dovecot MTA)

2013-11-11 Thread Jan-Frode Myklebust
My installation is only serving 1/10 of your size, but long time ago we
migrated off mysql for userdatabase, and over to LDAP. The MySQL data
source had issues (not dovecot related), and didn't seem like the right
tool for the job.

Initially we kept mysql as the authoritative database over our users, and
mirrored the user details over to LDAP/389ds -- which we pointed dovecot and
postfix to. Then eventually we migrated completely out of MySQL as user
database. LDAP/389ds gives us easy multimaster replication, easy
integration with dovecot, postfix, etc., client side support for
failover between servers, and it is very fast. I don't think we've ever
had any issue with the userdb after migrating to LDAP.

our two 389ds servers are doing about 80 ldap bind() authentications per
second (plus dovecot auth cache is masking a lot more), 300 searches/s
and are using about 20% of a single cpu core each.

So, I would very much recommend you look into if something similar can
work for you.



  -jf

On Mon, Nov 11, 2013 at 03:24:46PM +1000, Edwardo Garcia wrote:
 My company have 36 dovecots, one biggest ISP in country 3 million user,
 agree with Nick  poster, we had stop use dovecot load balance because too
 bad effect on primary database, now use single localhost, we have script
 run every 30 second to test login, if fail sleep 30 second, try again, fail
 and down ethernet interface so hardware load balancer see server not answer
 and can not use, nagios soon tell us of problem, very very bad and stupid
 way, but only option is safe, we have look at alternative to dovecot for
 this and still look, not happy with unreliable softwares to immitate
 feature.
 
 big network mean big time locate and fix problem when arise so you be good
 to say no extra point of failure. Too many cog in chain eventually lead to
 problem.
 
 Timo pleaz reconsider feature
 
 
 On Sun, Nov 10, 2013 at 4:21 PM, Nick Edwards nick.z.edwa...@gmail.comwrote:
 
  On 11/9/13, Timo Sirainen t...@iki.fi wrote:
   On 9.11.2013, at 5.11, Nick Edwards nick.z.edwa...@gmail.com wrote:
  
   On 11/9/13, Michael Kliewe mkli...@gmx.de wrote:
   Hi Timo,
  
   I would also, like others, see you mainly working on Dovecot as an IMAP
   server. As far as I can see there are many things on the roadmap, and I
   hope many more will be added (for example a built-in health-checker for
   director backends).
  
   Only if you have enough personal resources and Dovecot as an IMAP
  server
   will not loose your attention, I would love to see your expertise in
   making a better MTA.
  
   Yes, some of us have been waiting for some years now, for a
   configurable change to alter the method of dovecots method of
   failover, which is just load balancing between servers rather than
   true failover, like postix, I see now why it gets no importance.
  
   Ah, you’re talking about SQL connections. Had to look up from old mails
  what
   you were talking about. It hasn’t changed, because I think the current
   behavior with load balancing + failover is more useful than
  failover-only.
   And you can already do failover-only with an external load balancer.
  Sure,
   Dovecot could also implement it, but it’s not something I especially
  want to
   spend time on implementing.
  
 
  My employer has 18 pop3 servers, one imap customer access (imap here
  has so little use we cant justify a redundant machine, not for 11,
  yes, eleven only users after 2 years of offering imap , and 2 imap
  (webmail).
 
  Sp, each server has a replicated mysql database
 
  If I use your better method, I have 18 machines polling themselves
  and the MASTER server, this needlessly slams the daylights out of  the
  master as I'm sure even you can imagine.
 
  We have 4 customer relay smtp servers and 4 inbound smtp servers,
  postifx, using its failover and better method, means they only hit
  the master server when the local mysql unix socket is not listening,
  ie, mysqld  is stopped -  the master server NEVER sees them.
 
  How is your method, better than true failover like method used by
  postfix, your methods is load balancing, it is not failover, and
  causes problems on larger networks
 
  I'm sure in some cases most people using it are happy and wont have
  performance increases noticeable, but if you are going to offer a
  backup for auth, it really shoulds be able to configure, if we want it
  to DoS our master, or only talk to master when it cant talk local, so
  I think it should be matter you need to consider, else you are only
  half arsed doing it, and like implying we should go introduce a
  further point of failure, by using yet more third party softwares