Re: Scaling of imap servers

2011-06-15 Thread Bron Gondwana
On Wed, 15 Jun 2011 17:03:53 +0530
Ram  wrote:

> We need to create a platform for create a large number of cyrus accounts 
> that can scale indefinitely
> I could start with just 2k  users  but could have 20x the number of 
> accounts by next year.
> I was thinking of taking a cloud-based machine at amazon or rackspace 
> and scale hardware vertically  as required.
> But the problem with rackspace or amazon is they do not offer much 
> storage and cap the total storage that can be used.

Worry about IO too.  IO is your killer.

> So what is the best way of creating a scalable setup.

We're really happy with this config:

* 2U rack case
* twin server-grade CPU
* 48Gb RAM
* hardware raid controller with battery backup
* two SSD drives (64Gb plus) in RAID1
* 12 SATA 2Tb drives.

We create 5 x RAID1 volumes with two hotspares for the disks, giving
10Tb of usable space.  Total system cost is about US $13k per machine.

> If I use cyrus-murder still there is always a challenge of using a 
> single mupdate server which cannot handle more than "n"  requests at a 
> time.
> So I plan to use nginx proxy servers that will just redirect the 
> requests and use multiple servers behind it.
> For scaling I will have to add more servers behind the nginx , as long 
> as the proxy can support it.

That's what we do, it's great.

> Even though this is already working , but I dont see it as a long term 
> solution. Backups , HA , DR etc are all not very clean.
> 
> I would like to know how do you guys do the same

HA is overrated.  Manual failover can be achieved within a short time with
a bit of monitoring.

We use replication, because replication rocks.  Much better for both
minor disaster recovery and speed of failover than any backup based
recovery system.

That said, we have backups too.  Unfortunately the backup system is
still quite custom :(  But you can use more general backup system and
just take the additional IO hit.

Bron.

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Scaling of imap servers

2011-06-15 Thread Ram
We need to create a platform for create a large number of cyrus accounts 
that can scale indefinitely
I could start with just 2k  users  but could have 20x the number of 
accounts by next year.
I was thinking of taking a cloud-based machine at amazon or rackspace 
and scale hardware vertically  as required.
But the problem with rackspace or amazon is they do not offer much 
storage and cap the total storage that can be used.


So what is the best way of creating a scalable setup.

If I use cyrus-murder still there is always a challenge of using a 
single mupdate server which cannot handle more than "n"  requests at a 
time.
So I plan to use nginx proxy servers that will just redirect the 
requests and use multiple servers behind it.
For scaling I will have to add more servers behind the nginx , as long 
as the proxy can support it.

Even though this is already working , but I dont see it as a long term 
solution. Backups , HA , DR etc are all not very clean.

I would like to know how do you guys do the same



Thanks
Ram












Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: Cyrus 2.4.9beta2 Released

2011-06-15 Thread Lars Schimmer
On 2011-06-15 08:33, Bron Gondwana wrote:
> On Tue, 14 Jun 2011 20:46:44 -0700
> Craig White  wrote:

> I'm hoping that Debian will release 2.4.x packages into sid soon.  The
> main sticking point has been BDB upgrades - but I'm going to try to get
> a working BDB upgrader into 2.4.10.

Already in SID for some weeks now (at least 5 weeks yet. Setting up new 
server with it.

> Bron.
> 
> Cyrus Home Page: http://www.cyrusimap.org/
> List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


MfG,
Lars Schimmer
-- 
-
TU Graz, Institut für ComputerGraphik & WissensVisualisierung
Tel: +43 316 873-5405   E-Mail: l.schim...@cgv.tugraz.at
Fax: +43 316 873-5402   PGP-Key-ID: 0x4A9B1723

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/


Re: Cyrus 2.4.9beta2 Released

2011-06-15 Thread Lars Schimmer
On 2011-06-15 05:46, Craig White wrote:
> On Tue, 2011-06-14 at 23:02 +0200, Bron Gondwana wrote:

> speaking of this... anyone know where reasonable deb packages might be
> found or soon to be found? Ubuntu (Debian) is still on 2.2.x and I am
> migrating off from CentOS 5 (2.3.x) and the 'oneiric' solution was going
> to turn Ubuntu 10.04 (lucid) into Ubuntu bleeding edge (480 packages
> including db5.1 and the kitchen sink - a non-starter).>

Debian SID alreeady has 2.4.8:
apt-cache policy cyrus-common
cyrus-common:
   Installed: 2.4.8-8
   Candidate: 2.4.8-8
   Version table:
  *** 2.4.8-8 0
 150 http://ftp.tu-graz.ac.at/mirror/debian/ sid/main amd64 Packages
 100 /var/lib/dpkg/status

> Thanks and great to hear
>
> Craig
>
>


MfG,
Lars Schimmer
-- 
-
TU Graz, Institut für ComputerGraphik & WissensVisualisierung
Tel: +43 316 873-5405   E-Mail: l.schim...@cgv.tugraz.at
Fax: +43 316 873-5402   PGP-Key-ID: 0x4A9B1723

Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/