Hi, You say,
> The size of the users mail boxes is likely our biggest problem. A > previous manager refused to allow quotas, and now some users have mail > boxes up to 5 gigs. We are working on reducing it. Maby that's why is a "previous manager" I say "Impose Quotas because on day your database will explose" "and your going to be a previous manager" What I do is give my 2200 users 100% of there quota and tell them not to use more then 50% of there quotas. What I mean by that is that nobody knows how much space they have...... That way I can bring-up or bring-down anybody's quotas they don't know... but it works... and they clean up there accounts I have 2 quotas categories 80meg and 160meg but it could be 100meg and 200meg tomorrow if i want. I estimate that a users with a 80meg quotas can have over 2000 emails at 50%(40meg) with no attachement. Users can use 50% of theire quotas for attachements until they reach 100% by then thats there problem and they must put those attachement on there local drive or use pop3. Being a School Board nobody suppose to keep photos, mp3, jpeg ... etc on the School board mail server. Cordialement Jacques Beaudoin Agent d'administration Les services des technologies de l'information et des communications Commission scolaire de la Pointe de l'Île Montréal, Québec, Canada Courriel/Email: [EMAIL PROTECTED] Cel: 514 918-3350 Selon "M. J. [Mike] OBrien" <[EMAIL PROTECTED]>: > Sounds like you have your work cut out for you, Greg. The good part is that > you have an interim solution while you work out the best plan for your new > hardware. > > Especially upon learning you already use multiple hosts your experiences on > this one are surprising. You should be pretty good to run with your existing > resources. Consider moving the RDBMS completely on its own away from the > IMAP daemon -- as Paul was saying about the two being CPU hungry. Just think > of the flexibility you get from this. > > Final thought on "how clean is the database". Considering the history and > the user account sizes to 5-gig do you think this database has maybe scads > of disconnected (orphaned) messageblks etc.? There's a Database Cleanup > function in DBMA (http://dbma.ca) which fixes problems that emerged around > 2.02-2.04. Takes a long long time to run if there are a lot of messageblks > without mailboxes|users but the database gets a good cleaning. You say > dbmail-util chokes. Did the database evolve through upgrades of DBMail 2.02 > through 2.09? Could be it needs some pretty aggressive cleaning like maybe a > dump and reassemble. There are other ways to migrate too. > > Good luck... stay posted as you go and folks will likely be glad to lend a > thinking cap or two :o) > Mike > > > > ----- Original Message ----- > From: "Greg Hellings" <[EMAIL PROTECTED]> > To: "DBMail mailinglist" <[email protected]> > Sent: Monday, March 06, 2006 1:56 PM > Subject: Re: [Dbmail] Terrible Load Issues With 2.0.9 > > > > M. J. [Mike] OBrien wrote: > >> ( I see since I started this Mathew has dropped you a note about > >> PostGreSQL which is just so true. Sure some mysql folks don't like > >> hearing that but if you know pgsql .... it's not so scary) > > > > We picked mysql because we do want to replicate and cluster the database > > for redundancy and this seemed easier with mysql. I have new hardware > > coming > > in to do this. > > > > M. J. [Mike] OBrien wrote: > >> * dbmail.conf? > >> You are not using a socket connection for mysql? > >> You have likely experimented with changes to the number of child > >> processes in dbmail.conf but I do wonder on the basis of the small amount > >> of data here about max 200 IMAPD / 200 POP children and 1000 + 1000 > >> connections when you have only max connected 170 users shared between > >> POP3 and IMAP. I wonder which protocol users like the most? Are you > >> getting POP DoS'd? Many-users-POP-every-min kinda thing? Try jack up the > >> mysql query cache to something huge. > > > > We are only using imap, no pop. Sorry I should have said that. We > > actually split the > > imap off on to two machine as well. Half our clients are connecting to > > the same machine > > the SQL server is running on, and the other half are running on another > > machine. > > > > > > M. J. [Mike] OBrien wrote: > >> After many hours of uptime what is the RES and actual memory used for > >> MySQL and how much mem does DBMail really get after all the buffers and > >> such get their chunk? ( After a 7 or 14-day uptime the used MYSQL memory > >> should be your new configged RES plus a little -- giving the rest back to > >> the OS, the Raid 5 drive buffers(!), MTA and MDA.) > >> > > > > The services are never up for that long. Restarting the services > > regularly has been helping performance greatly. The dbmail-imapd process > > is being restart on both machines once and hour and I'm manually > > restarting mysql at least a few times a day (trying different settings). > > > > > > M. J. [Mike] OBrien wrote: > >> * Is the database clean? > >> That's a large MySQL database for only 300 users. There's nothing crazy > >> saying even a terrabyte db can run well with MySQL but it is somewhat new > >> paradigm and takes a bit of science and mad luck to get the right formula > >> of resources and config to do it well -- I think about your scaling > >> dilemma. Meanwhile for 300 megs and more I favour PostgreSQL if I want to > >> do it the easy way. With MySQL you have to figure out how to do it. With > >> pgsql you just do it. But... truth is, DBMail works vewry well with > >> MySQL. > > > > Jacques Beaudoin wrote: > >> Why is your dbmail_messageblk so large ? > >> > >> Cordialement > >> > > > > The size of the users mail boxes is likely our biggest problem. A > > previous manager refused to allow quotas, and now some users have mail > > boxes up to 5 gigs. We are working on reducing it. > > > > > > M. J. [Mike] OBrien wrote: > >> You are using MySQL 4.1? I hope. > > > > We were. I was having the same problems with 4.1. We had also been > > running on a 32 OS. This was a rush job to fix a down mail server > > situation. Later we switched out the OS to a 64 bit OS and upgraded to > > Mysql 5.0 by mistake (no one had any sleep). > > > > > > M. J. [Mike] OBrien wrote: > >> Is the database clean? Each of your users must be storing about .7 Gigs > >> mail -- a full cd's worth. Send each user a blank CD and tell them to get > >> their crap off your server :o) heh heh don't. How often do you have cron > >> run dbmail-util -a -y ? > > > > This is setup to run nightly, but it is failing. All the sub commands run > > by themselves except the optimize, which fails after running for a very > > long time. We are running all other sub commands nightly. > > > > > > M. J. [Mike] OBrien wrote: > >> Are you pushing into swap? Is 4G the max for that architecture? I feel > >> that is quite light for the size of chunks you seem to be using (700megs > >> per user manipulated by 170 connections at peak.) Your mysql conf file on > >> a rough count is for something a tad larger like 2.5 gig per cpu. Are > >> those 32-bit Xeons? > > > > We are not swapping > > > > vmstat 1 10 > > procs -----------memory---------- ---swap-- -----io---- --system-- > > ----cpu---- > > r b swpd free buff cache si so bi bo in > > cs us sy id wa > > 1 3 308 28124 12144 995388 0 0 251 213 48 13 4 6 85 > > 5 > > 4 2 308 51940 11864 983356 0 0 10016 208 2443 4253 49 14 > > 17 21 > > 2 2 308 42176 11928 991500 0 0 10084 2152 4475 6278 43 25 > > 14 19 > > 1 3 308 66412 11888 983332 0 0 10544 552 4077 7021 38 19 > > 20 22 > > 1 3 308 44348 11916 987408 0 0 4412 124 2593 4601 37 15 > > 32 16 > > 7 0 308 27824 11956 999680 0 0 9448 2548 5580 10111 52 27 > > 8 13 > > 2 1 308 50932 12012 1001676 0 0 9712 9496 3740 8081 49 27 > > 9 16 > > 2 4 308 36644 11972 995560 0 0 9356 4348 3834 5623 51 27 > > 11 11 > > 4 7 308 45024 11964 997620 0 0 4872 216 2108 3168 54 14 > > 3 29 > > 2 2 308 57996 11568 989808 0 0 8016 6072 2634 7302 58 28 > > 9 5 > > > > M. J. [Mike] OBrien wrote: > >> I think thread_concurrency=8 is tricky with Hyperthreading. Is HTT off or > >> on. I know a lot of people think this is really four cpus but MySQL is > >> smarter maybe cuz I found that thread_concurrency=4 worked slightly > >> better with 32-bit Hyperthreading "ON". > >> > > > > It's On. > > > > > > Paul Stevens wrote: > >> What does the slow query log teach you? What kind of imap commands do > >> your clients typically use? In my experience, imap search can easily > >> cause the kind of problems you're describing. > >> > >> Consider splitting the mysql server and the imap frontend server(s); > >> since both are cpu hungry you're likely to suffer thread trashing and/or > >> excessive context switching. > > > > It's hard to read at this point because all kinds of queries are running > > slow. The whole system is over loaded. > > > > > > I have three new machines that just came in today to replace the system > > this is on. A number of things were wrong with the system it is on, many > > have been pointed out here, but I didn't think any of them were bad enough > > to cause these load averages. The three machines we just got will be > > behind an LVS, they each have 6G of ram, dual 3.0 Xeons with 2M L2 cache, > > RAID 1 with 5 146GB SCSIs. What I'm worried is that I will install all of > > this and the load will scale up. > > > > This is what I'm seeing right now > > uptime > > 10:43:39 up 21:51, 1 user, load average: 19.28, 16.49, 13.96 > > > > > > Thanks so much for all your insightful responses. I can't tell you how > > much I appreciate it. > > > > > > -- > > Greg Hellings Network & Security Analyst > > [EMAIL PROTECTED] Dailey & Associates > > (310) 279-4321 Fax (310) 360-0810 > > _______________________________________________ > > Dbmail mailing list > > [email protected] > > https://mailman.fastxs.nl/mailman/listinfo/dbmail > > > > _______________________________________________ > Dbmail mailing list > [email protected] > https://mailman.fastxs.nl/mailman/listinfo/dbmail >
