Hello Daryl,
On 2014-01-30 13:37, Darryl Sutherland wrote:
We are investigating to see if mailpiler is a suitable replacement
for MailArchiva. A few weeks ago I made a feature request to extend
piler's authentication to generic ldap, and this feature was released
in 2 days. This great responsiveness on your part was very well
received, and the generic ldap authentication is working quite well.
Ahh, I remember, I'm glad to hear that
However, because mail ingestion and client churn generate a lot of
activity in ISP scale environments, a single server installation would
incur heavy database and disk activity when provisioning and purging
data for new and cancelled clients. What is the recommended
architecture to scale piler up to "20 000 users and beyond"?
so far the biggest piler installation I'm aware of is for a company
with
~4000 users, ~10 million emails, and roughly 1 TB stored data.
How would you architect Mailpiler to provide highly scalable
capacity, performance and availability? Bearing in mind the possible
impact of ingestion and data purging on both the file system and the
database. Do you have any examples or case studies on clients from
large enterprises who have moved to mailpiler, and how did they meet
the architectural requirements?
the company mentioned above has provided a single virtual machine for
piler
to archive emails. (Actually they have another piler VM at another
datacenter,
just in case the primary site goes down, but these two VMs are
independent).
I just saw your email archiving product line on your site, and if you
offer
10+ years of retention for 20k users (and perhaps more), then we
definitely
need a distributed solution where the huge data is somehow separated,
either
based on functionality (sphinx, mysql, emails) or user ids, or even
years.
However I haven't been part of such a huge project yet, so I don't have
much
experience with the area (yet), although I have some (yet unproved)
ideas.
If you have some time and resources to experiment with piler, I would
work
together with you to come up with an enhanced version of piler that
supports
such a huge setup you have. However expect a somewhat longer ETA than 2
days
to finish it :-)
For starters what hardware infrastructure will you provide for a piler
based
email archiving? Would it be a virtualized environment with a shared
storage
like netapp? Or several computers with some local storage? How many
users
perform searches at the same time?
Btw. just for being curious, what's the layout of the current
mailarchiva
based system for the load you described?
Best regards,
Janos
