> Henrik K <h...@hege.li> kirjoitti 5.12.2018 kello 18.27:
>
>
> Is it scanning the last two files, are you at 90%+? How many entries in
> *.log compared to your corpus? I guess you could compare those to find out
> what the "missing" files are and try scanning those manually to see if they
> hang..
>
Now it is running as expected as I implemented the changes you suggested. At
least it survives now and if the output tells anything from the timeouts I will
see and report.
Br. jarif
>
> On Wed, Dec 05, 2018 at 05:35:56PM +0200, Jari Fredriksson wrote:
>> It looks this now: http://snpy.in/27fice
>>
>>
>>> Henrik K <h...@hege.li> kirjoitti 5.12.2018 kello 17.11:
>>>
>>>
>>> Try running with PERL_SIGNALS=unsafe env, I believe that should allow alarm
>>> to abort running regexes.
>>>
>>> PERL_SIGNALS=unsafe bin/automasscheck-minimal.sh
>>>
>>> You can also set some lower time_limit than default 300 in .automasscheck.cf
>>>
>>> run_all_masschecks() {
>>> run_masscheck single-corpus --all --pre "time_limit 30" \
>>>
>>> Timeouted messages should be found with
>>>
>>> grep TIME_LIMIT_EXCEEDED *.log
>>>
>>>
>>> On Wed, Dec 05, 2018 at 06:14:40AM -0800, Steven Ihde wrote:
>>>> My corpus is similar in size, but it hangs around 30%.
>>>>
>>>> I took a backtrace with gdb this morning; result below. I think it's
>>>> just spinning in a regex match, but this is the first time I've looked
>>>> at a perl interpreter backtrace. Could somebody have added a test with a
>>>> bad regex? Is there any way to enable some debug logging for the mass
>>>> check script so we can tell which test is hanging?
>>>>
>>>> (gdb) info threads
>>>> Id Target Id Frame
>>>> * 1 Thread 0x7f550d8f82c0 (LWP 477) "perl" 0x0000564bab22b16b in ?? ()
>>>> (gdb) t
>>>> [Current thread is 1 (Thread 0x7f550d8f82c0 (LWP 477))]
>>>> (gdb) bt
>>>> #0 0x0000564bab22b16b in ?? ()
>>>> #1 0x0000564bab238eb3 in Perl_regexec_flags ()
>>>> #2 0x0000564bab1c358c in Perl_pp_match ()
>>>> #3 0x0000564bab1bf7a6 in Perl_runops_standard ()
>>>> #4 0x0000564bab145775 in perl_run ()
>>>> #5 0x0000564bab11e9fd in main ()
>>>>
>>>>
>>>> On 12/5/18 01:59, Tom Hendrikx wrote:
>>>>> Hi,
>>>>>
>>>>> I have a relatively small sample set, this is the output of yesterdays
>>>>> run:
>>>>>
>>>>> status: starting scan stage now:
>>>>> 2018-12-04 10:00:22 AM
>>>>> status: completed scan stage, 4560 messages now:
>>>>> 2018-12-04 10:00:24 AM
>>>>> status: starting run stage now:
>>>>> 2018-12-04 10:00:24 AM
>>>>> status: 10% ham: 397 spam: 60 date: 2012-12-18 now:
>>>>> 2018-12-04 10:01:29 AM
>>>>> status: 20% ham: 793 spam: 121 date: 2014-04-09 now:
>>>>> 2018-12-04 10:02:39 AM
>>>>> status: 30% ham: 1191 spam: 180 date: 2018-10-26 now:
>>>>> 2018-12-04 10:03:54 AM
>>>>> status: 40% ham: 1592 spam: 236 date: 2016-02-04 now:
>>>>> 2018-12-04 10:05:35 AM
>>>>> status: 50% ham: 1993 spam: 292 date: 2016-10-03 now:
>>>>> 2018-12-04 10:07:04 AM
>>>>> status: 60% ham: 2388 spam: 354 date: 2017-03-30 now:
>>>>> 2018-12-04 10:08:21 AM
>>>>> status: 70% ham: 2788 spam: 411 date: 2018-11-22 now:
>>>>> 2018-12-04 10:09:12 AM
>>>>> status: 80% ham: 3184 spam: 472 date: 2018-03-11 now:
>>>>> 2018-12-04 10:11:11 AM
>>>>> status: 90% ham: 3579 spam: 534 date: 2018-07-24 now:
>>>>> 2018-12-04 10:14:32 AM
>>>>> status: completed run stage now:
>>>>> 2018-12-04 11:45:05 PM
>>>>>
>>>>> Note that the progress was ok until the last 10%, where it took 13.5h
>>>>> in stead of a few minutes. Definitely something is broken here...
>>>>>
>>>>> Kind regards,
>>>>> Tom
>>>>>
>>>>> On 04-12-18 21:53, Kevin A. McGrail wrote:
>>>>>> Yes, Jarif was saying the same thing is happening to him. I just
>>>>>> cross posted his email to the system admins. Might need to trace
>>>>>> back to changes for the past few days.
>>>>>>
>>>>>> Regards,
>>>>>> KAM
>>>>>> --
>>>>>> Kevin A. McGrail
>>>>>> VP Fundraising, Apache Software Foundation
>>>>>> Chair Emeritus Apache SpamAssassin Project
>>>>>> https://www.linkedin.com/in/kmcgrail - 703.798.0171
>>>>>>
>>>>>>
>>>>>> On Tue, Dec 4, 2018 at 3:41 PM Steven Ihde <si...@hamachi.us
>>>>>> <mailto:si...@hamachi.us>> wrote:
>>>>>>
>>>>>> Anybody else having trouble with this? Second (third?) morning in
>>>>>> a row
>>>>>> I've found perl processing spinning at 100% CPU, hours after they
>>>>>> should
>>>>>> have been done, and killed them. Based on the output looks like they
>>>>>> both hung around 30% of the way through. If it happens again I'll
>>>>>> try to
>>>>>> debug a little to figure out where they are hanging.
>>>>>>
>>>>>
>>>
>
