Re: [clamav-users] Question about LLVM...

2018-12-12 Thread J.R. 
> So I would like to ask, does bytecode have access to its environment
> (like ActiveX unfortunately did) and, how well is bytecode sandboxed?

Well, first of all, only bytecode signatures published by Cisco/Talos
are considered "trusted" and will run by default. You would have to
manually specify if you wanted to run unsigned bytecode signatures.

>From what I've read, the bytecode is C-like, but it is limited in that
it can't access system calls or memory, can only access the file to be
scanned, it does have an internal timeout, and other security measures
to prevent it from arbitrarily doing what it wants.

You can always look through the source code if you want.

It doesn't seem like the bytecode database gets updated very often. I
suppose it is reserved for complex scanning when the pattern matching
of the regular databases just won't cut it...
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Question about LLVM...

2018-12-12 Thread Paul Kosinski 
I've always been leery of executable code that gets downloaded "behind
the scenes" and then executed for whatever purpose. In the "old days",
people were warned against downloading random software and then
executing it. How that's become at least half of what we do on a daily
basis -- in our browsers!

The most obvious current example of this is Javascript, now that Java,
Flash and ActiveX (but not PDF), have been almost killed off in
browsers. (And this is why I'm also a big fan of NoScript!)

So I would like to ask, does bytecode have access to its environment
(like ActiveX unfortunately did) and, how well is bytecode sandboxed? 

P.S. One of main the reasons I moved to Open Source ClamAV was that 
traditional commercial solutions (like Symantec) were not only opaque
but also stuck their fingers into various parts of the underlying OS
(which ClamAV doesn't do).



On Tue, 11 Dec 2018 13:30:12 -0500
Scott Kitterman  wrote:

> On Tuesday, December 11, 2018 05:59:05 PM Micah Snyder wrote:
> > Sorry about the broken links on the website and in the clamav-faq
> > manual pages.  Our web dev team is actively working on integrating
> > the newly remodeled user manual into the website.
> > 
> > The bytecode interpreter was nonfunctional for a long time but was
> > fixed a few years ago. This is why LLVM was prioritized over the
> > bytecode compiler.
> > 
> > Functionally, from an outside perspective, the feature set of using
> > bytecode interpreter vs LLVM is the same. The cost/benefit analysis
> > of LLVM-JIT vs Interpreter hinges on whether or not executing
> > native code is sufficiently faster than interpreting the bytecodes
> > to outweigh the cost of JIT compilation. Our bytecode signatures
> > themselves are relatively small and are relatively few, so the
> > advantage of executing native code vs the time lost JIT compiling
> > the bytecode is, I'm told, negligible. The developers who did the
> > initial benchmarking on the subject have since left the team and
> > while I've been told that the performance is "about the same", I
> > don't have any figures to back up that up. If anyone out there
> > decides to do additional research on the subject, do note that
> > bytecode functions are only executed for certain file types, so
> > benchmark findings will vary by file type.
> > 
> > The TL;DR is that we're not aware of any significant advantage of
> > using LLVM over the bytecode interpreter at this time.
> > 
> > Regarding the reason for only supporting older versions of LLVM:
> > It takes time to update to use newer APIs.  The LLVM project has
> > been moving pretty fast and we simply haven't prioritized dev and
> > test time towards updating our LLVM support.  In fact, Debian
> > provides a patch to ClamAV to support LLVM 3.7-3.9, but we haven't
> > had the time to properly integrate and test it.  Because the
> > bytecode interpreter is working so well, we're focusing our efforts
> > on other tasks.
> 
> And unfortunately the developer who was doing that work in Debian has
> moved on to other things, so we won't be providing patches for later
> versions.
> 
> Might it make sense in the next feature release to just kill off LLVM
> and move on.  That would certainly help with clarity and focus.
> 
> Scott K

> 
> 
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Question about LLVM...

2018-12-11 Thread J.R. 
Micah & Scott,

Thank you for the replies, you answered exactly what I was thinking
too based on posts referring to the built-in improvements and hush on
llvm.
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Question about LLVM...

2018-12-11 Thread Scott Kitterman 
On Tuesday, December 11, 2018 05:59:05 PM Micah Snyder wrote:
> Sorry about the broken links on the website and in the clamav-faq manual
> pages.  Our web dev team is actively working on integrating the newly
> remodeled user manual into the website.
> 
> The bytecode interpreter was nonfunctional for a long time but was fixed a
> few years ago. This is why LLVM was prioritized over the bytecode compiler.
> 
> Functionally, from an outside perspective, the feature set of using bytecode
> interpreter vs LLVM is the same. The cost/benefit analysis of LLVM-JIT vs
> Interpreter hinges on whether or not executing native code is sufficiently
> faster than interpreting the bytecodes to outweigh the cost of JIT
> compilation. Our bytecode signatures themselves are relatively small and
> are relatively few, so the advantage of executing native code vs the time
> lost JIT compiling the bytecode is, I'm told, negligible. The developers
> who did the initial benchmarking on the subject have since left the team
> and while I've been told that the performance is "about the same", I don't
> have any figures to back up that up. If anyone out there decides to do
> additional research on the subject, do note that bytecode functions are
> only executed for certain file types, so benchmark findings will vary by
> file type.
> 
> The TL;DR is that we're not aware of any significant advantage of using LLVM
> over the bytecode interpreter at this time.
> 
> Regarding the reason for only supporting older versions of LLVM:  It takes
> time to update to use newer APIs.  The LLVM project has been moving pretty
> fast and we simply haven't prioritized dev and test time towards updating
> our LLVM support.  In fact, Debian provides a patch to ClamAV to support
> LLVM 3.7-3.9, but we haven't had the time to properly integrate and test
> it.  Because the bytecode interpreter is working so well, we're focusing
> our efforts on other tasks.

And unfortunately the developer who was doing that work in Debian has moved on 
to other things, so we won't be providing patches for later versions.

Might it make sense in the next feature release to just kill off LLVM and move 
on.  That would certainly help with clarity and focus.

Scott K
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

Re: [clamav-users] Question about LLVM...

2018-12-11 Thread Micah Snyder (micasnyd) 
Sorry about the broken links on the website and in the clamav-faq manual pages. 
 Our web dev team is actively working on integrating the newly remodeled user 
manual into the website.

The bytecode interpreter was nonfunctional for a long time but was fixed a few 
years ago. This is why LLVM was prioritized over the bytecode compiler.

Functionally, from an outside perspective, the feature set of using bytecode 
interpreter vs LLVM is the same. The cost/benefit analysis of LLVM-JIT vs 
Interpreter hinges on whether or not executing native code is sufficiently 
faster than interpreting the bytecodes to outweigh the cost of JIT compilation. 
Our bytecode signatures themselves are relatively small and are relatively few, 
so the advantage of executing native code vs the time lost JIT compiling the 
bytecode is, I'm told, negligible. The developers who did the initial 
benchmarking on the subject have since left the team and while I've been told 
that the performance is "about the same", I don't have any figures to back up 
that up. If anyone out there decides to do additional research on the subject, 
do note that bytecode functions are only executed for certain file types, so 
benchmark findings will vary by file type.

The TL;DR is that we're not aware of any significant advantage of using LLVM 
over the bytecode interpreter at this time.

Regarding the reason for only supporting older versions of LLVM:  It takes time 
to update to use newer APIs.  The LLVM project has been moving pretty fast and 
we simply haven't prioritized dev and test time towards updating our LLVM 
support.  In fact, Debian provides a patch to ClamAV to support LLVM 3.7-3.9, 
but we haven't had the time to properly integrate and test it.  Because the 
bytecode interpreter is working so well, we're focusing our efforts on other 
tasks.

Regards,
Micah

Micah Snyder
ClamAV Development
Talos
Cisco Systems, Inc.


On Dec 11, 2018, at 10:05 AM, J.R. 
mailto:themadbea...@gmail.com>> wrote:

I've googled to no end, but haven't been able to come up with anything
except a few snips mentioning LLVM and bytecode here and there...

I'm curious exactly what the benefit would be to use LLVM, is there
much of a performance gain over the built-in (non-llvm) bytecode
interpreter? Is it an expanded feature set? Why the limitation of
using only such old versions of LLVM?

The last time I looked at the manual it only mentioned compilation
options, and that's it... The current link to the ClamAV manual is
broken on the website too, fyi... :(

Not complaining, just curious...
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml

[clamav-users] Question about LLVM...

2018-12-11 Thread J.R. 
I've googled to no end, but haven't been able to come up with anything
except a few snips mentioning LLVM and bytecode here and there...

I'm curious exactly what the benefit would be to use LLVM, is there
much of a performance gain over the built-in (non-llvm) bytecode
interpreter? Is it an expanded feature set? Why the limitation of
using only such old versions of LLVM?

The last time I looked at the manual it only mentioned compilation
options, and that's it... The current link to the ClamAV manual is
broken on the website too, fyi... :(

Not complaining, just curious...
___
clamav-users mailing list
clamav-users@lists.clamav.net
http://lists.clamav.net/cgi-bin/mailman/listinfo/clamav-users


Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq

http://www.clamav.net/contact.html#ml