Can I please confirm - according the upgrade page it would appear that I need 
to load the new WebUI and run the upgrade script for the database. With this in 
mind, is it also necessary to recompile? And should that be done before or 
after the DB upgrade is completed?

Ross Gailer
Communications Manager
Direct Phone: 02 6770 1743

> On 18 Oct. 2016, at 7:37 am, Janos SUTO <> wrote:
> Dear piler users,
> I've just released the latest stable version of piler. Actually it has been 
> released
> a while ago, I just had no time to write this email.
> Notice that it's 1.2.x, and not 1.1.x, that is there're some minor 
> incompatibilities
> you must be aware of. I've compiled a RELEASE_NOTES file which describes some 
> of the
> changes.
> The most important change is that I've moved all piler related configs to 
> ${sysconfdir}/piler
> directory (with the default options it's /usr/local/etc/piler).
> It means that whatever you had in /usr/local/etc, that must be moved to 
> /usr/local/etc/piler,
> eg. /usr/local/etc/piler.conf -> /usr/local/etc/piler/piler.conf, etc.
> I've decided to put the sphinx config file to  ${sysconfdir}/piler. Debian 
> and Ubuntu ship
> a sphinx package which enabled a periodic indexer --all cron job, which 
> practically destroys
> the sphinx indices, and despite both the install docs and the FAQ warn about 
> it, many piler
> users fell for this debian 'trick'.
> To match the new path, I've updated the rc.searchd file, and the indexer 
> shell scripts as well.
> If you upgraded, then be sure to run the util/db-upgrade-1.1.0-vs-1.2.0.sql 
> script. If you have
> questions about the upgrade procedure, don't hesitate to ask. I recommend you 
> to run pilerconf
> after the upgrade, and check if you get the values in piler.conf back. If so, 
> then the config
> files are at the proper new location.
> What next? I have three interesting topics in my head. One of them is high 
> availability.
> Currently your most basic option is to setup two archives (even in different 
> locations),
> and have your mail server send copies of emails to both archives. Then you 
> have two
> independent archives with the same content. Either of them goes down, your 
> archived
> emails are still accessible.
> However it's not that elegant, and while this approach may work out for you, 
> it can be
> improved. Mysql supports a cluster mode. Sphinx data can be replicated easily 
> (think about
> rsync), however replicating the stored millions of files is not that easy. 
> I've seen some
> replicating object stores, eg. swift from openstack or ambry from linkedin. I 
> think they
> could be used to replicate the stored encrypted and gzipped files.
> Another idea in my head is zstandard. It's facebook's new compressing 
> algorithm which
> outperforms gzip in every way. Fortunately it can read gzipped data (=your 
> already
> stored emails will be readable in the future), and new emails can be 
> compressed with
> zstandard's new algorithm offering better speed and compression.
> The 3rd thing in my head is a non forking version of piler. An o365 user 
> reported a
> problem that he got lots of NDRs of undeliverable emails. It turned out that 
> o365
> has no means of flow control, so in case of a spike in the email volume the 
> default
> 10 piler workers are not enough to handle the emails delivered in paralel. 
> After
> a trial and error approach it required 40 piler workers to serve the load.
> A non forking piler smtp server would solve the problem by only receiving the 
> emails,
> and amazingly fast. With such a processing model it can receive 100 or even 
> more
> smtp sessions simultaneously very effectively. Then we need a few workers that
> actually processing the stored emails, ie. parsing, indexing, encrypting and
> storing.
> I'm investigating the poll() mechanism at the moment. I've been told to use 
> epoll,
> because it's much more efficient than select() or poll(), however epoll is 
> Linux only.
> So if anyone used piler on freebsd, solaris, etc. other than Linux, then it 
> would
> be a problem. So before picking either poll or epoll, let me know if someone
> uses piler on a Unix flavor other than Linux.
> Let me know what you think.
> Janos


Presbyterian Ladies' College Armidale
Locked Bag 5
ABN:79 017 381 796
CRICOS provider code: 02295G
Tel:    +61 2 6770 1700 
Fax:  + 61 2 6770 1797

In the interests of environmental sustainability, PLC is endeavouring to 
communicate electronically where possible.

The contents of this email are confidential. Any unauthorised use of the 
contents is expressly prohibited. If you have received this email in error, 
please advise by telephone (reverse charges) immediately and then 
delete/destroy the email and any printed copies. Thank you.

Reply via email to