Re: [exim] Advice: NFS, hardware, SATA vs SAS etc
On 2019-12-04, venbian via Exim-users wrote: > Hello gurus, > > As 2020 nears I wanted to ask opinions about the current state of > hardware requirements for a small business email platform. $dayjob > asked me to enhance our existing platform to improve performance and > add redundancy. Q1: how much of each? > Main questions (TLDR): > > Can NFS handle heavy IMAP, LDA, HTTP workload? yes: this is not microsoft - there are no software performance limits. But using NFS for server disk will double your network traffic (or worse when doing server side searches) > Is direct attached SATA III 6Gb/s SSD in RAID 1 sufficient or is SAS needed? see Q1 > Is gigabit ethernet the bottleneck in any case? I can upgrade to a > 10 gigabit local network if advisable. > Can NFS peacefully co-exist with other mail system workloads without > resource contention such as SQL DB or spamassassin or redis, etc? if your database is on direct attached disk then yes. > More detail: > > I had bad experiences with NFS approx 15yrs ago where IMAP load > saturated controller link (yes, noatime was used on the mount) and was > unusable. But in 2020 is it time to give NFS another look? why would you want to use NFS? I would pick E-SATA or USB3 direct-attach over NFS, and internal direct-attach over that. > Current mail storage setup uses local attached large SATA SSD and >does well, but it directly hosts HTTP, LDA, IMAP and Submission which >could all be faster and it only does nightly backups. Adding SAN is >probably out of $dayjob pricerange and SAS is borderline. Power >consumption is also a factor so instead of a dedicated file server I >thought it would make more sense to build a big server with direct >attached fast SATA SSD in mirrored RAID that also has strong CPU and >maximum memory so it can also run some of the backend process such as >spamassassin, redis or SQL database etc. (we want to start using SQL >DB for more which means it will be under heavy use) > What workloads can best co-exist with NFS where each does not contend for the > other's resources? NFS server uses disk and LAN, D L NFS client uses RAM and LAN, R L S.A. uses CPU and RAM,R C Redis uses RAM and CPUR C SQL uses RAM and disk and CPU D R C Redis and SQL also use the lan if they're serving other servers. > I'd put a couple smaller machines in HA in front of that to proxy > webmail, HTTP website, IMAP and Submission. Edge MTA is on a separate > server and would probably stay that way, maybe adding a failover. It > keeps a fraction of its mail in the local system but will make more > heavy use of the SQL DB which I thought to also put on the file > server(?) > Could NFS keep up with load for proxy of HTTP, IMAP, LDA, etc? > > Is local attached SATA SSD in RAID 1 ok? Will 6Gb/s SATA III be a bottleneck > in any possible scenario? I was looking at motherboards with multiple PCIe or > M.2 slots thinking NVMe bandwidth (3GB/s) would be great but I'm unsure if > NFS, gigabit ethernet or other components could even make use of it. see Q1 > Or is that too amateur and local attached (hope not remote attached) SAS a > minimum requirement? SAS SSDs are a newer thing I never used and expensive > for the $boss. I read some people express doubts that SSD is suited for SAS > at all which is one reason I thought just use SATA for more efficient power > and cost. > > Our workload: > > We process a lot of mail but as you can guess, don't have tremendous > storage needs. We have several tens of thousands of users but a > smaller fraction of that are actively using mail every day. Maildir > storage is several TB. Exact daily mail volume is unknown but should > be on the order of a few million, many which users have forwarded to > to other accounts so a small fraction is stored locally. Forwarding can get messy when gmail thinks _you_ are spamming. SATA will probably work alright there. > We also have few TB of web data that is hosted from a server in the > same location that I thought to unify into the NFS setup. I wouldn't use NFS as server disk unless the service was CPU intensive. so servers running Redis or S.A. are good candidates for being NFS clients. others not so much. -- When I tried casting out nines I made a hash of it. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] av_scanner and panic log...
Mandi! Jeremy Harris via Exim-users In chel di` si favelave... >> But every time clamd have trouble, exim log in panic_log, and so debian >> compain about it. > That's the point of the panic log. It's a place where real problems > get put, so that you can notice. Debian is complaining because > clamd is complaining. Ok, i try to restate better. ;-) Why exim consider an av_scanner failure as an error for panic_log? Why simply does not defer the message? There's some way to 'lower' the error priority of av_scanner? The question born by the fact that clamav is indeed a bed beast nowadays: while updating the signatures (expecially on low-end machine) does not respond, exim go in timeout and add a panic log. I've tried now: av_scanner = clamd:/var/run/clamav/clamd.ctl retry=5s and i'll increase timeout to see if helps. Thanks. -- ...di una locomotiva, come cosa viva lanciata a bomba contro l'ingiustizia (F. Guccini) -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
Re: [exim] Advice: NFS, hardware, SATA vs SAS etc
Admin note follows: On 4 Dec 2019, at 19:34, venbian via Exim-users wrote: > As 2020 nears I wanted to ask opinions about the current state of hardware > requirements for a small business email platform. $dayjob asked me to enhance > our existing platform to improve performance and add redundancy. I allowed this one through as there's a lot of experience of these matters on this mailing list, however I would ask that you reply to the OP directly rather than the list. Thanks Graeme -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
[exim] Advice: NFS, hardware, SATA vs SAS etc
Hello gurus, As 2020 nears I wanted to ask opinions about the current state of hardware requirements for a small business email platform. $dayjob asked me to enhance our existing platform to improve performance and add redundancy. Main questions (TLDR): Can NFS handle heavy IMAP, LDA, HTTP workload? Is direct attached SATA III 6Gb/s SSD in RAID 1 sufficient or is SAS needed? Is gigabit ethernet the bottleneck in any case? I can upgrade to a 10 gigabit local network if advisable. Can NFS peacefully co-exist with other mail system workloads without resource contention such as SQL DB or spamassassin or redis, etc? More detail: I had bad experiences with NFS approx 15yrs ago where IMAP load saturated controller link (yes, noatime was used on the mount) and was unusable. But in 2020 is it time to give NFS another look? Current mail storage setup uses local attached large SATA SSD and does well, but it directly hosts HTTP, LDA, IMAP and Submission which could all be faster and it only does nightly backups. Adding SAN is probably out of $dayjob pricerange and SAS is borderline. Power consumption is also a factor so instead of a dedicated file server I thought it would make more sense to build a big server with direct attached fast SATA SSD in mirrored RAID that also has strong CPU and maximum memory so it can also run some of the backend process such as spamassassin, redis or SQL database etc. (we want to start using SQL DB for more which means it will be under heavy use) What workloads can best co-exist with NFS where each does not contend for the other's resources? I'd put a couple smaller machines in HA in front of that to proxy webmail, HTTP website, IMAP and Submission. Edge MTA is on a separate server and would probably stay that way, maybe adding a failover. It keeps a fraction of its mail in the local system but will make more heavy use of the SQL DB which I thought to also put on the file server(?) Could NFS keep up with load for proxy of HTTP, IMAP, LDA, etc? Is local attached SATA SSD in RAID 1 ok? Will 6Gb/s SATA III be a bottleneck in any possible scenario? I was looking at motherboards with multiple PCIe or M.2 slots thinking NVMe bandwidth (3GB/s) would be great but I'm unsure if NFS, gigabit ethernet or other components could even make use of it. Or is that too amateur and local attached (hope not remote attached) SAS a minimum requirement? SAS SSDs are a newer thing I never used and expensive for the $boss. I read some people express doubts that SSD is suited for SAS at all which is one reason I thought just use SATA for more efficient power and cost. Our workload: We process a lot of mail but as you can guess, don't have tremendous storage needs. We have several tens of thousands of users but a smaller fraction of that are actively using mail every day. Maildir storage is several TB. Exact daily mail volume is unknown but should be on the order of a few million, many which users have forwarded to to other accounts so a small fraction is stored locally. We also have few TB of web data that is hosted from a server in the same location that I thought to unify into the NFS setup. Thank you for reading and your insight. -- ## List details at https://lists.exim.org/mailman/listinfo/exim-users ## Exim details at http://www.exim.org/ ## Please use the Wiki with this list - http://wiki.exim.org/
