Running MySQL 5.6.24
It sounds to me like you've got sumpin busted.
Has it always been this way?
Have you checked that all perl modules are up to date (I doubt that's it)
Really, I'd consider cloning this machine, maybe spin it up in a VM, and
trying new perl and new ASSP. A pain to test that way, but better than
buggering up your production machine....
On Wed, Mar 16, 2016 at 6:49 PM, Jay <h...@herodata.com> wrote:
> So this day keeps on getting weirder as time goes on today. I think I
> really need to go back to bed and start the day over tomorrow.
>
> So, looking at my log for ASSP today, since I turned off DoHMM and still
> using MySQL for spamdb, ASSP has not had to perform a restart until 6PM
> today. Where as previously our ASSP installation would have to restart
> several times a day because of the 1.8GB memory usage limit on the version
> of Perl we are using. Perl.exe would just die when it hit the wall at 1.8GB
> of memory usage. The way I know the amount of restarts is by looking
> through our maillog.txt file I would find several listings where [startup]
> Starting as a service would be logged.
>
> So this brings me to ever lasting question now, what is making MySQL and
> ASSP so slow to be ready? I know HMM is the more precise and efficient
> method for detecting spam. I really want to use it, but the question arises
> now, why would my memory usage be so high when using HMM and cause the
> problem we ran into this morning. Looking at ASSP now with only the spamdb
> in usage, it started within 15-20 seconds. Where as previously it was
> several minutes before ASSP was ready when HMM was used in conjunction with
> the spamdb.
>
> I could upgrade MySQL from 5.1.45(current) to 5.5.48 but I am not totally
> convinced this will be the answer to the issue we dealt with. Could it be
> that my HMM table is corrupt?
>
> K Post, what version of MySQL are you running?
>
>
> On 3/16/2016 2:30 PM, K Post wrote:
>
>> I'm no expert here, but I don't think your slow startup has to do with the
>> perl version.
>>
>> My Perl came right from ActiveState.
>> https://www.activestate.com/activeperl/downloads
>> If you downoad the 64 bit version, it really installs the 32 bit version
>> with the 64 bit flag from what I can tell.
>>
>> If you have a dev / staging server, you might try upgrading to 2.4.7. I
>> know at one point Thomas changed the order of startup so that SMTP started
>> sooner than the webui. I don't know if your version/build has that change
>> or not, but I'm pretty sure it's in the latest 2.4.7 - it's definitely in
>> the current dev versions.
>>
>>
>>
>>
>> On Wed, Mar 16, 2016 at 1:30 PM, Jay <h...@herodata.com> wrote:
>>
>> Here's the machine's specs: Windows 7 Pro 64bit, 16GB ram and Core-i7 3770
>>> 8core 3.4Ghz processor.
>>>
>>> Looking at the version of perl you have running, it has the 64bit flag
>>> turned on. Which is what I am looking for as well. The version of Perl I
>>> am
>>> running is just plain 32bit. It does not have the 64bit flag turned on. I
>>> saw a build for Strawberry Perl that has that flag enabled. ActiveState
>>> as
>>> far as my search has found is that they do not do custom builds with that
>>> 64bit flag turned on.
>>>
>>> Currently this mail server has 45 domains and 550 users. So it's pretty
>>> busy. I have not seen at any point any of the 8 cores get even close to
>>> being maxed. On average, maybe 5% utilization for CPU. Out of the 16gb
>>> ram,
>>> about 43-45% is utilized at any point.
>>>
>>> Perl -v shows this on our version:
>>> perl 5, version 16, subversion 3 (v5.16.3) built for
>>> MSWin32-x86-multi-thread
>>>
>>> So no 64bit flag turned on when it was compiled and distributed by
>>> ActiveState.
>>>
>>>
>>> On 3/16/2016 12:24 PM, K Post wrote:
>>>
>>> Just for reference:
>>>> Pretty small user base.
>>>> spamdb 61mb, 900k rows
>>>> HMMdb 390mb 2.7 million rows
>>>>
>>>> perl 5, version 20, subversion 1 (v5.20.1) built for
>>>> MSWin32-x86-multi-thread-64int running on 64bit Windows 2012r2
>>>>
>>>> We have almost no startup delay, but we're running 2.4.8 (dev).
>>>>
>>>> Perl uses about 950mb ram.
>>>>
>>>> Our machine isn't particularly powerful, either. Is there any chance
>>>> that
>>>> yours is underpowered?
>>>>
>>>> On Wed, Mar 16, 2016 at 10:30 AM, Jay <h...@herodata.com> wrote:
>>>>
>>>> Good day everyone.
>>>>
>>>>> I am hoping someone can give me some guidance on this. Here's some info
>>>>> and the issue I am dealing with:
>>>>>
>>>>> We are using ASSP 2.4.5(15272) with ActiveState Perl 5.16.3 (32bit). We
>>>>> also use MySQL(5.1.45) as our database back end. The only data that we
>>>>> send through MySQL is the spamdb and HMMdb.
>>>>>
>>>>> Here's some info on MySQL, currently HMMdb is 826mb and has almost
>>>>> 5.4million rows. Our spamdb is currently at 237mb and has almost 1.9
>>>>> million rows. We have a lot more users on our mail server now than we
>>>>> did when we started using ASSP which has been in place for several
>>>>> years
>>>>> now. Close to 8 years at this point.
>>>>>
>>>>> Problems:
>>>>> 1. So the first issue I am dealing with is when we have to restart
>>>>> ASSP,
>>>>> it can take several minutes for ASSP to be ready because it appears the
>>>>> hold up at this point is MySQL. MySQL is slow to send the info needed
>>>>> to
>>>>> ASSP and thus holds up the SMTP processing of email. I realize this and
>>>>> am planning on making whatever changes I need to get this resolved.
>>>>>
>>>>> 2. Trying to use the spamdb and HMMdb files resident in memory at this
>>>>> point is an issue because we are using Perl 32bit. Once the perl
>>>>> service
>>>>> hits 1.8GB of ram it crashes instantly. I researched this and it is a
>>>>> know problem with Perl 32bit unless USE_64_BIT_INT enabled and compiled
>>>>> in that version of Perl. I want to use the HMM and spamdb in
>>>>> conjunction with each other.
>>>>>
>>>>> 3. Strawberry Perl has a build specially for still using a 32bit build
>>>>> but have the USE_64_BIT_INT enabled in the build so 32bit Perl can use
>>>>> 3GB of ram total. The other alternative is to use 64bit Perl, but I
>>>>> remember in the past trying this and ASSP would just crash within a few
>>>>> seconds after starting. Reading through the previous knowledge base
>>>>> there was at one point the discussion that not all modules on CPAN are
>>>>> compiled in 64bit. So the consensus was to use 32bit Perl. Is this
>>>>> still
>>>>> the case anymore or does it make sense to move to Perl 64bit?
>>>>>
>>>>> So I realize that MySQL is not really cutting it for us any longer. We
>>>>> are out growing it quickly. For now I turned off DoHMM so we don't have
>>>>> to wait for several minutes for ASSP to start up. Once we disabled
>>>>> DoHMM, ASSP starts within 30 seconds. This is only a temporary measure
>>>>> until I can get some guidance on what to do about Perl. Anyone out
>>>>> there
>>>>> using either the special build of Strawberry Perl with the
>>>>> USE_64_BIT_INT enabled? Or successfully using 64bit Perl itself?
>>>>>
>>>>> Please let me know and thank you any assistance. I really appreciate
>>>>> it.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> ------------------------------------------------------------------------------
>>>>> Transform Data into Opportunity.
>>>>> Accelerate data analysis in your applications with
>>>>> Intel Data Analytics Acceleration Library.
>>>>> Click to learn more.
>>>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>>>>> _______________________________________________
>>>>> Assp-user mailing list
>>>>> Assp-user@lists.sourceforge.net
>>>>> https://lists.sourceforge.net/lists/listinfo/assp-user
>>>>>
>>>>>
>>>>>
>>>>
>>>> ------------------------------------------------------------------------------
>>>> Transform Data into Opportunity.
>>>> Accelerate data analysis in your applications with
>>>> Intel Data Analytics Acceleration Library.
>>>> Click to learn more.
>>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>>>>
>>>>
>>>> _______________________________________________
>>>> Assp-user mailing list
>>>> Assp-user@lists.sourceforge.net
>>>> https://lists.sourceforge.net/lists/listinfo/assp-user
>>>>
>>>>
>>>> -----
>>>> No virus found in this message.
>>>> Checked by AVG - www.avg.com
>>>> Version: 2016.0.7497 / Virus Database: 4542/11821 - Release Date:
>>>> 03/16/16
>>>>
>>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Transform Data into Opportunity.
>>> Accelerate data analysis in your applications with
>>> Intel Data Analytics Acceleration Library.
>>> Click to learn more.
>>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>>> _______________________________________________
>>> Assp-user mailing list
>>> Assp-user@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/assp-user
>>>
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Transform Data into Opportunity.
>> Accelerate data analysis in your applications with
>> Intel Data Analytics Acceleration Library.
>> Click to learn more.
>> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
>>
>>
>> _______________________________________________
>> Assp-user mailing list
>> Assp-user@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/assp-user
>>
>>
>> -----
>> No virus found in this message.
>> Checked by AVG - www.avg.com
>> Version: 2016.0.7497 / Virus Database: 4542/11821 - Release Date: 03/16/16
>>
>
>
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
> _______________________________________________
> Assp-user mailing list
> Assp-user@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/assp-user
>
>
------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785231&iu=/4140
_______________________________________________
Assp-user mailing list
Assp-user@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/assp-user
