In that case you should load the rules from file in the main thread and then pass the YR_RULES pointer to the other threads. If you load the rules from each thread you end up with a lot of copies of YR_RULES which is wasteful. It can be done, and it will be fine if your have only a few rules, but if you have a lot of them you will be wasting memory.
Yes, you can increase MAX_THREADS and rebuild the library. On Fri, Sep 29, 2017 at 12:00 AM, Igor Polevoy <[email protected]> wrote: > Oh, I see MAX_THREADS is 32. But there should be no problem to redefine it > to a bigger number and rebuild the library, right? > > > On Thursday, 28 September 2017 14:48:29 UTC-7, Igor Polevoy wrote: >> >> Hi Victor, >> thx a lot for the prompt response. >> >> Do you mean that all the threads must have the same YR_RULES for a reason >> (other than memory savings)? >> I actually will need to call yr_rules_load_file, as we have our rules >> stored in the file at some moment before even the main thread started... >> But there is of course NP to call it from the main, just curious. >> >> On Thursday, 28 September 2017 14:13:01 UTC-7, Víctor Manuel Álvarez >> García wrote: >>> >>> That flow is not correct, when you call yr_rules_load() you get a new >>> YR_RULES struct, so the threads are not sharing the same rules, each thread >>> has its own copy (although they are identical because they were loaded from >>> the same file). What you should do is something like: >>> >>> main thread: >>> yr_initialize() >>> yr_compiler_create() >>> yr_compiler_add_string/file/fd() >>> yr_compiler_get_rules() >>> yr_compiler_destroy() >>> >>> * span threads and pass them the YR_RULES* pointer you got from >>> yr_compiler_get_rules(). Notice that you can't span more than MAX_THREADS >>> (defined in libyara/include/yara/limits.h) >>> * wait for all threads to finish >>> >>> yr_finalize() >>> >>> >>> scan threads: >>> >>> yr_scan_mem() >>> yr_finalize_thread() >>> >>> You can take a look at yara.c which uses a multithreaded approach while >>> scanning directories. >>> >>> Regards, >>> Víctor >>> >>> >>> >>> On Thu, Sep 28, 2017 at 8:41 PM, Igor Polevoy <[email protected]> >>> wrote: >>> >>>> Hi >>>> I need to port an old code from YARA C API for v 1.7 to 3.6 >>>> The usage model is multi-threaded, with each thread using the same >>>> rules to scan different files. >>>> So I just want to understand if the relevant YARA code is reentrant and >>>> what should be the correct flow? >>>> >>>> Let say I have in the *main thread* : >>>> * yr_initialize()* >>>> * yr_compiler_create()* >>>> * ....* >>>> * yr_rules_save(file)* >>>> * yr_compiler_destroy()* >>>> ... >>>> /* spawn the threads here */ >>>> >>>> And in each one of *thread functions*: >>>> *yr_rules_load(file)* >>>> *yr_scan_mem()* >>>> *yr_finalize_thread()* >>>> >>>> Is it the correct flow or do I miss anything? >>>> >>>> Many thanks >>>> Igor >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "YARA" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- > You received this message because you are subscribed to the Google Groups > "YARA" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "YARA" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
