Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-19 Thread Micah Snyder (micasnyd) 
I'm not sure that I can provide a more satisfactory answer.  The built-in 
interpreter executes bytecode operations more slowly but doesn't have a 
compilation step.  LLVM compiles the bytecodes signatures down to machine 
language and executes it quickly, but that compile step makes the speed boost a 
bit of wash because the bytecode signatures are for the most part not very 
large functions.

At this time, I'm not able to recommend one over the other, with the exception 
that using LLVM adds an extra dependency.

Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.


On Jul 19, 2018, at 4:23 AM, Sergey mailto:a_...@sama.ru>> wrote:

On Tuesday 17 July 2018, Micah Snyder (micasnyd) wrote:

If you don't provide the older LLVM 3.6 for ClamAV, it will
use it's built-interpreter rather than just-in-time-compile
the signatures.

b.t.w. Can you describe differences between built-interpreter
and LLVM in short ? Which is more preferable to use ?

--
Regards, Sergey
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-19 Thread Sergey 
On Tuesday 17 July 2018, Micah Snyder (micasnyd) wrote:

> If you don't provide the older LLVM 3.6 for ClamAV, it will
> use it's built-interpreter rather than just-in-time-compile
> the signatures.   
 
b.t.w. Can you describe differences between built-interpreter
and LLVM in short ? Which is more preferable to use ?

-- 
Regards, Sergey
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Sergey 
On Tuesday 17 July 2018, Micah Snyder (micasnyd) wrote:

> If you don't provide the older LLVM 3.6 for ClamAV, it will use
> it's built-interpreter rather than just-in-time-compile the signatures.  

Thanks.

-- 
Regards,
Sergey
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Sierk Bornemann 


> Am 17.07.2018 um 15:47 schrieb Micah Snyder (micasnyd) :
> 
> You're making an assumption that the LLVM 3.7-3.9 patches are ready-to-take.

Debian seems to use it since a while on all its stable and unstable branches...

> Last time I worked with them I had some issues with the patches on systems 
> other than Debian.

OK. And why not ironed out the concerning glichtes or given them a priority to 
iron that out?

> At the time, we were attempting to wrap up a _very_ long development cycle 
> with final bug fixes and regression testing.  We decided it was more 
> important to get the release out.  The LLVM patches were pushed to the next 
> release (aka 0.101).

OK

> For reference, our Bugzilla ticket to apply the LLVM 3.7, 3.8, 3.9 patches is 
> here.  Please bare in mind if you read the ticket that our product versioning 
> changed. Our previous lead didn't recognize a need for security/patch 
> releases.  0.99.3 and 0.99.4 ended up being security patch releases.  In the 
> ticket, "0.99.3" refers to 0.100, and "0.99.4" refers to 0.101:
> https://bugzilla.clamav.net/show_bug.cgi?id=11869

I know. I am already CC’d to this ticket, which also is provided with LLVM 
3.7/3.8/3.9-patches since 2017-07-07 by Sebastian A. Siewior, which 
unfortunately so far didn’t make it into the upstream sources.


Regards,
Sierk Bornemann

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Micah Snyder (micasnyd) 
You're making an assumption that the LLVM 3.7-3.9 patches are ready-to-take.  
Last time I worked with them I had some issues with the patches on systems 
other than Debian.  At the time, we were attempting to wrap up a _very_ long 
development cycle with final bug fixes and regression testing.  We decided it 
was more important to get the release out.  The LLVM patches were pushed to the 
next release (aka 0.101).

For reference, our Bugzilla ticket to apply the LLVM 3.7, 3.8, 3.9 patches is 
here.  Please bare in mind if you read the ticket that our product versioning 
changed. Our previous lead didn't recognize a need for security/patch releases. 
 0.99.3 and 0.99.4 ended up being security patch releases.  In the ticket, 
"0.99.3" refers to 0.100, and "0.99.4" refers to 0.101:
https://bugzilla.clamav.net/show_bug.cgi?id=11869


Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.


On Jul 17, 2018, at 9:15 AM, Sierk Bornemann 
mailto:sie...@gmx.de>> wrote:



Am 17.07.2018 um 14:44 schrieb Micah Snyder (micasnyd) 
mailto:micas...@cisco.com>>:

ClamAV has 3 options for handling bytecode signatures:
• Built-in LLVM (based on LLVM 2.8)
• Built-in bytecode interpreter
• System-installed LLVM (support limited to LLVM 3.6 at this time, although 
Debian has had success with a set of patches that enabled support up to 3.9).

Concerning your last item „Debian has had success with a set of patches that 
enabled support up to 3.9", you mean

https://salsa.debian.org/clamav-team/clamav/blob/stretch/debian/patches/Add-support-for-LLVM-3.7.patch
https://salsa.debian.org/clamav-team/clamav/blob/stretch/debian/patches/Add-support-for-LLVM-3.8.patch
https://salsa.debian.org/clamav-team/clamav/blob/stretch/debian/patches/Add-support-for-LLVM-3.9.patch

from https://salsa.debian.org/clamav-team/clamav/tree/stretch/debian/patches?

And why not _at least_ take these successful and ready-to-take patches and 
merge them by the clamav team into the official clamav upstream sources to _at 
least_ officially support LLVM up to version 3.9?


Regards,
Sierk Bornemann

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Sierk Bornemann 


> Am 17.07.2018 um 14:44 schrieb Micah Snyder (micasnyd) :
> 
> ClamAV has 3 options for handling bytecode signatures:
>   • Built-in LLVM (based on LLVM 2.8)
>   • Built-in bytecode interpreter
>   • System-installed LLVM (support limited to LLVM 3.6 at this time, 
> although Debian has had success with a set of patches that enabled support up 
> to 3.9).

Concerning your last item „Debian has had success with a set of patches that 
enabled support up to 3.9", you mean

https://salsa.debian.org/clamav-team/clamav/blob/stretch/debian/patches/Add-support-for-LLVM-3.7.patch
https://salsa.debian.org/clamav-team/clamav/blob/stretch/debian/patches/Add-support-for-LLVM-3.8.patch
https://salsa.debian.org/clamav-team/clamav/blob/stretch/debian/patches/Add-support-for-LLVM-3.9.patch

from https://salsa.debian.org/clamav-team/clamav/tree/stretch/debian/patches?

And why not _at least_ take these successful and ready-to-take patches and 
merge them by the clamav team into the official clamav upstream sources to _at 
least_ officially support LLVM up to version 3.9?


Regards,
Sierk Bornemann
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Micah Snyder (micasnyd) 
ClamAV has 3 options for handling bytecode signatures:

  1.  Built-in LLVM (based on LLVM 2.8)
  2.  Built-in bytecode interpreter
  3.  System-installed LLVM (support limited to LLVM 3.6 at this time, although 
Debian has had success with a set of patches that enabled support up to 3.9).

With 0.99 the built-in LLVM was preferred over the bytecode interpreter.
With 0.100, the built-in LLVM (2.8) feature was deprecated in favor of either 
the interpreter or system-installed LLVM (when available).  It's still there, 
but we are hoping to remove it in a future version.

If you don't provide the older LLVM 3.6 for ClamAV, it will use it's 
built-interpreter rather than just-in-time-compile the signatures.


Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.


On Jul 17, 2018, at 6:05 AM, Sergey mailto:a_...@sama.ru>> wrote:

On Tuesday 17 July 2018, Al Varnell wrote:

It's best to use the bytecode interpreter for ClamAV
bytecode signatures, but if for some reason you feel
you must use LLVM-JIT

I thought it was necessary to use llvm to use bytecode
signatures. Was I wrong? Is ClamAV not lost functionality
without LLVM?

--
Regards, Sergey
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Sergey 
On Tuesday 17 July 2018, Al Varnell wrote:

> It's best to use the bytecode interpreter for ClamAV
> bytecode signatures, but if for some reason you feel
> you must use LLVM-JIT  

I thought it was necessary to use llvm to use bytecode
signatures. Was I wrong? Is ClamAV not lost functionality
without LLVM?

-- 
Regards, Sergey
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Al Varnell 
On Tue, Jul 17, 2018 at 02:03 AM, Sergey wrote:
> On Tuesday 17 July 2018, Al Varnell wrote:
> 
>>> Yes. But LLVM < 3.7 deprecated also.
>> 
>> No, deprecated refers to the use of LLVM greater than 3.6 by
>> ClamAV 0.100.0, not LLVM itself.
> 
> Deprecated refers to use deprecated LLVM 3.6 which can be absent
> in modern distro. Therefore deprecating internal LLVM code support
> is not good idea I think.


Correct. It's not a good idea. It's best to use the bytecode interpreter for 
ClamAV bytecode signatures, but if for some reason you feel you must use 
LLVM-JIT then it has to be v3.6 or below and it still may not comprise on all 
platforms.

-Al-
-- 
Al Varnell
Mountain View, CA





___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Sergey 
On Tuesday 17 July 2018, Al Varnell wrote:

> > Yes. But LLVM < 3.7 deprecated also.
> 
> No, deprecated refers to the use of LLVM greater than 3.6 by
> ClamAV 0.100.0, not LLVM itself.

Deprecated refers to use deprecated LLVM 3.6 which can be absent
in modern distro. Therefore deprecating internal LLVM code support
is not good idea I think.

-- 
Regards, Sergey
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Al Varnell 
On Tue, Jul 17, 2018 at 01:34 AM, Sergey wrote:
> On Tuesday 17 July 2018, tschmidt wrote:
> 
 ClamAV 0.100.0 has been released!
>>> 
  *   Deprecating internal LLVM code support. The configure script has 
 changed to search the system for an installed instance of the LLVM 
 development libraries, and to otherwise use the bytecode interpreter for 
 ClamAV bytecode signatures. To use the LLVM Just-In-Time compiler for 
 executing bytecode signatures, please ensure that the LLVM development 
 package at version 3.6 or lower is installed. Using the deprecated LLVM 
 code is possible with the command: ./configure --with-system-llvm=no, but 
 it no longer compiles on all platforms.
>>> 
>>> 
>>> Hm. But 3.x updated to 3.8 about 2 years ago.
>> 
>> That's the meaning of the word "deprecating".
> 
> Yes. But LLVM < 3.7 deprecated also.

No, deprecated refers to the use of LLVM greater than 3.6 by ClamAV 0.100.0, 
not LLVM itself.

-Al-
-- 
Al Varnell
Mountain View, CA





___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Sergey 
On Tuesday 17 July 2018, tschmidt wrote:

> >> ClamAV 0.100.0 has been released!
> > 
> >>   *   Deprecating internal LLVM code support. The configure script has 
> >> changed to search the system for an installed instance of the LLVM 
> >> development libraries, and to otherwise use the bytecode interpreter for 
> >> ClamAV bytecode signatures. To use the LLVM Just-In-Time compiler for 
> >> executing bytecode signatures, please ensure that the LLVM development 
> >> package at version 3.6 or lower is installed. Using the deprecated LLVM 
> >> code is possible with the command: ./configure --with-system-llvm=no, but 
> >> it no longer compiles on all platforms.
> > 
> > 
> > Hm. But 3.x updated to 3.8 about 2 years ago.
> 
> That's the meaning of the word "deprecating".

Yes. But LLVM < 3.7 deprecated also.

-- 
Regards, Sergey
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread tschmidt 
Am 17.07.2018 um 09:47 schrieb Sergey:
> On Monday 09 April 2018, Joel Esler (jesler) wrote:
> 
>> ClamAV 0.100.0 has been released!
> 
>>   *   Deprecating internal LLVM code support. The configure script has 
>> changed to search the system for an installed instance of the LLVM 
>> development libraries, and to otherwise use the bytecode interpreter for 
>> ClamAV bytecode signatures. To use the LLVM Just-In-Time compiler for 
>> executing bytecode signatures, please ensure that the LLVM development 
>> package at version 3.6 or lower is installed. Using the deprecated LLVM code 
>> is possible with the command: ./configure --with-system-llvm=no, but it no 
>> longer compiles on all platforms.
> 
> 
> Hm. But 3.x updated to 3.8 about 2 years ago.

That's the meaning of the word "deprecating".

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-07-17 Thread Sergey 
On Monday 09 April 2018, Joel Esler (jesler) wrote:

> ClamAV 0.100.0 has been released!

>   *   Deprecating internal LLVM code support. The configure script has 
> changed to search the system for an installed instance of the LLVM 
> development libraries, and to otherwise use the bytecode interpreter for 
> ClamAV bytecode signatures. To use the LLVM Just-In-Time compiler for 
> executing bytecode signatures, please ensure that the LLVM development 
> package at version 3.6 or lower is installed. Using the deprecated LLVM code 
> is possible with the command: ./configure --with-system-llvm=no, but it no 
> longer compiles on all platforms.


Hm. But 3.x updated to 3.8 about 2 years ago.

-- 
Regards, Sergey
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] ClamAV® blog: ClamAV 0.100.0 has been released!

2018-04-09 Thread Joel Esler (jesler) 


https://blog.clamav.net/2018/04/clamav-01000-has-been-released.html

ClamAV 0.100.0 has been released!
Join us as we welcome ClamAV 0.100.0 to the family officially.  You can grab 
it, as always, from the downloads page on 
ClamAV.net.

ClamAV 0.100.0 is a feature release which includes many code submissions from 
the ClamAV community.  Some of the more prominent submissions include:



  *   Interfaces to the Prelude SIEM open source package for collecting ClamAV 
virus events.
  *   Support for Visual Studio 2015 for Windows builds.  Please note that we 
have deprecated support for Windows XP, and while Vista may still work, we no 
longer test ClamAV on Windows XP or Vista.
  *   Support libmspack internal code or as a shared object library. The 
internal library is the default and includes modifications to enable parsing of 
CAB files that do not entirely adhere to the CAB file format.
  *   Linking with OpenSSL 1.1.0.
  *   Deprecation of the AllowSupplementaryGroups parameter statement in clamd, 
clamav-milter, and freshclam. Use of supplementary is now in effect by default.
  *   Numerous bug fixes, typo corrections, and compiler warning fixes.


Additionally, we have introduced important changes and new features in ClamAV 
0.100, including but not limited to:



  *   Deprecating internal LLVM code support. The configure script has changed 
to search the system for an installed instance of the LLVM development 
libraries, and to otherwise use the bytecode interpreter for ClamAV bytecode 
signatures. To use the LLVM Just-In-Time compiler for executing bytecode 
signatures, please ensure that the LLVM development package at version 3.6 or 
lower is installed. Using the deprecated LLVM code is possible with the 
command: ./configure --with-system-llvm=no, but it no longer compiles on all 
platforms.
  *   Compute and check PE import table hash (a.k.a. "imphash") signatures.
  *   Support file property collection and analysis for MHTML files.
  *   Raw scanning of PostScript files.
  *   Fix clamsubmit to use the new virus and false positive submission web 
interface.
  *   Optionally, flag files with the virus "Heuristic.Limits.Exceeded" when 
size limitations are exceeded.
  *   Improved decoders for PDF files.
  *   Reduced number of compile time warnings.
  *   Improved support for C++11.
  *   Improved detection of system installed libraries.
  *   Fixes to ClamAV's Container system and the introduction of Intermediates 
for more descriptive signatures.
  *   Improvements to clamd's On-Access scanning capabilities for Linux.


Acknowledgements

The ClamAV team thanks the following individuals for their code submissions:



  *   Andreas Schulze
  *   Anthony Chan
  *   Bill Parker
  *   Chris Miserva
  *   Daniel J. Luke
  *   Georgy Salnikov
  *   James Ralston
  *   Jonas Zaddach
  *   Keith Jones
  *   Marc Deslauriers
  *   Mark Allan
  *   Matthew Boedicker
  *   Michael Pelletier
  *   Ningirsu
  *   Sebastian Andrzej Siewior
  *   Stephen Welker
  *   Tuomo Soini


Known Issues

ClamAV has an active issue queue and enjoys continual improvement but as sad as
 I am to say it, we couldn't address every bug in this release.  I want to draw
 your attention a couple bugs in particular so as not to frustrate users
 setting up ClamAV:



  *   Platform: macOS:
 *   Bug:  If you attempt to build ClamAV with a system installed LLVM you 
may receive a linker error.  We recently changed default linking behavior to 
prefer dynamic linking over static linking.  As a result, we've uncovered a bug 
in building on macOS where dynamic linking against the LLVM libraries fails.  
To work around this bug, please add the --with-llvm-linking=static option to 
your ./configure call.




  *   Platform: CentOS 6 32bit, older versions of AIX:
 *   Bug:  On CentOS 6 32bit we observed that specific versions of zlib 
fail to correctly decompress the CVD signature databases.  If you are on an 
older system such as CentoOS 6 32bit and observe failures loading the signature 
database, please consider upgrading to a newer version of zlib.




  *   Platform: Miscellaneous
 *   Bug:  When cross compiling on certain legacy systems (Solaris, AIX, 
OSX) against older system libraries that do not support strn functions linking 
may fail during compile time. While automatic checking is done during configure 
time to check for unsupported libs, this problem can be manually avoided using 
the --enable-strni configure flag if it is encountered.
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml