The answer to your first question is no. It has not always been this way.
This is really starting to look like the HMMdb table in MySQL might be
corrupt/damaged in some way. I looked through my rebuild logs from the
past 6 months and what I am finding is before this situation happened,
our HMM rebuild was taking from 10-15 mins tops to finish according to
the log. Now looking at the log when we started having the slow issue,
our HMM rebuild is taking 30mins or longer to finish rebuilding. Now I
realize as the data grows it's going to take longer for a rebuild, but
there's a major jump in the rebuild for the HMM is a very short time
frame. Within a couple of days and not a huge jump in data size.
So with the information I uncovered and thinking this over, I am going
to go at this in a few steps:
1. Update MySQL to 5.5.48
2. Delete the HMM table out of MySQL and start over fresh
At that point I will see where things stand. I appreciate all the input
on this and advice.
Thank you.
On 3/16/2016 8:04 PM, K Post wrote:
Running MySQL 5.6.24
It sounds to me like you've got sumpin busted.
Has it always been this way?
Have you checked that all perl modules are up to date (I doubt that's it)
Really, I'd consider cloning this machine, maybe spin it up in a VM, and
trying new perl and new ASSP. A pain to test that way, but better than
buggering up your production machine....
On Wed, Mar 16, 2016 at 6:49 PM, Jay <h...@herodata.com> wrote:
So this day keeps on getting weirder as time goes on today. I think I
really need to go back to bed and start the day over tomorrow.
So, looking at my log for ASSP today, since I turned off DoHMM and still
using MySQL for spamdb, ASSP has not had to perform a restart until 6PM
today. Where as previously our ASSP installation would have to restart
several times a day because of the 1.8GB memory usage limit on the version
of Perl we are using. Perl.exe would just die when it hit the wall at 1.8GB
of memory usage. The way I know the amount of restarts is by looking
through our maillog.txt file I would find several listings where [startup]
Starting as a service would be logged.
So this brings me to ever lasting question now, what is making MySQL and
ASSP so slow to be ready? I know HMM is the more precise and efficient
method for detecting spam. I really want to use it, but the question arises
now, why would my memory usage be so high when using HMM and cause the
problem we ran into this morning. Looking at ASSP now with only the spamdb
in usage, it started within 15-20 seconds. Where as previously it was
several minutes before ASSP was ready when HMM was used in conjunction with
the spamdb.
I could upgrade MySQL from 5.1.45(current) to 5.5.48 but I am not totally
convinced this will be the answer to the issue we dealt with. Could it be
that my HMM table is corrupt?
K Post, what version of MySQL are you running?
On 3/16/2016 2:30 PM, K Post wrote:
I'm no expert here, but I don't think your slow startup has to do with the
perl version.
My Perl came right from ActiveState.
https://www.activestate.com/activeperl/downloads
If you downoad the 64 bit version, it really installs the 32 bit version
with the 64 bit flag from what I can tell.
If you have a dev / staging server, you might try upgrading to 2.4.7. I
know at one point Thomas changed the order of startup so that SMTP started
sooner than the webui. I don't know if your version/build has that change
or not, but I'm pretty sure it's in the latest 2.4.7 - it's definitely in
the current dev versions.
On Wed, Mar 16, 2016 at 1:30 PM, Jay <h...@herodata.com> wrote:
Here's the machine's specs: Windows 7 Pro 64bit, 16GB ram and Core-i7 3770
8core 3.4Ghz processor.
Looking at the version of perl you have running, it has the 64bit flag
turned on. Which is what I am looking for as well. The version of Perl I
am
running is just plain 32bit. It does not have the 64bit flag turned on. I
saw a build for Strawberry Perl that has that flag enabled. ActiveState
as
far as my search has found is that they do not do custom builds with that
64bit flag turned on.
Currently this mail server has 45 domains and 550 users. So it's pretty
busy. I have not seen at any point any of the 8 cores get even close to
being maxed. On average, maybe 5% utilization for CPU. Out of the 16gb
ram,
about 43-45% is utilized at any point.
Perl -v shows this on our version:
perl 5, version 16, subversion 3 (v5.16.3) built for
MSWin32-x86-multi-thread
So no 64bit flag turned on when it was compiled and distributed by
ActiveState.
On 3/16/2016 12:24 PM, K Post wrote:
Just for reference:
Pretty small user base.
spamdb 61mb, 900k rows
HMMdb 390mb 2.7 million rows
perl 5, version 20, subversion 1 (v5.20.1) built for
MSWin32-x86-multi-thread-64int running on 64bit Windows 2012r2
We have almost no startup delay, but we're running 2.4.8 (dev).
Perl uses about 950mb ram.
Our machine isn't particularly powerful, either. Is there any chance
that
yours is underpowered?
On Wed, Mar 16, 2016 at 10:30 AM, Jay <h...@herodata.com> wrote:
Good day everyone.
I am hoping someone can give me some guidance on this. Here's some info
and the issue I am dealing with:
We are using ASSP 2.4.5(15272) with ActiveState Perl 5.16.3 (32bit). We
also use MySQL(5.1.45) as our database back end. The only data that we
send through MySQL is the spamdb and HMMdb.
Here's some info on MySQL, currently HMMdb is 826mb and has almost
5.4million rows. Our spamdb is currently at 237mb and has almost 1.9
million rows. We have a lot more users on our mail server now than we
did when we started using ASSP which has been in place for several
years
now. Close to 8 years at this point.
Problems:
1. So the first issue I am dealing with is when we have to restart
ASSP,
it can take several minutes for ASSP to be ready because it appears the
hold up at this point is MySQL. MySQL is slow to send the info needed
to
ASSP and thus holds up the SMTP processing of email. I realize this and
am planning on making whatever changes I need to get this resolved.
2. Trying to use the spamdb and HMMdb files resident in memory at this
point is an issue because we are using Perl 32bit. Once the perl
service
hits 1.8GB of ram it crashes instantly. I researched this and it is a
know problem with Perl 32bit unless USE_64_BIT_INT enabled and compiled
in that version of Perl. I want to use the HMM and spamdb in
conjunction with each other.
3. Strawberry Perl has a build specially for still using a 32bit build
but have the USE_64_BIT_INT enabled in the build so 32bit Perl can use
3GB of ram total. The other alternative is to use 64bit Perl, but I
remember in the past trying this and ASSP would just crash within a few
seconds after starting. Reading through the previous knowledge base
there was at one point the discussion that not all modules on CPAN are
compiled in 64bit. So the consensus was to use 32bit Perl. Is this
still
the case anymore or does it make sense to move to Perl 64bit?
So I realize that MySQL is not really cutting it for us any longer. We
are out growing it quickly. For now I turned off DoHMM so we don't have
to wait for several minutes for ASSP to start up. Once we disabled
DoHMM, ASSP starts within 30 seconds. This is only a temporary measure
until I can get some guidance on what to do about Perl. Anyone out
there
using either the special build of Strawberry Perl with the
USE_64_BIT_INT enabled? Or successfully using 64bit Perl itself?
Please let me know and thank you any assistance. I really appreciate
it.
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7497 / Virus Database: 4542/11821 - Release Date:
03/16/16
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
-----
No virus found in this message.
Checked by AVG - www.avg.com
Version: 2016.0.7497 / Virus Database: 4542/11821 - Release Date: 03/16/16
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
