The commit history is messed up between 0.100 and 0.101 due to old (bad) commit
cherry-picking practices back then. That commit was also in 0.100, here:
https://github.com/Cisco-Talos/clamav-devel/commit/28592e59091ba353e637a7cde1038be1e426274b
Ignore the 0.99.3 branch name. The 0.99.3 feature dev branch was renamed to
0.100 to make space for security patch releases after Steve left.
-Micah
> -----Original Message-----
> From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On Behalf Of
> Andrew Williams
> Sent: Tuesday, March 9, 2021 4:21 PM
> To: ClamAV Development <clamav-devel@lists.clamav.net>
> Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1
>
> Mark,
>
> It looks like this commit, which according to the GitHub tags was introduced
> in
> ClamAV 0.101-beta, made it so that .ign2 rules could no longer have '.{}' on
> the
> end
>
> https://github.com/Cisco-Talos/clamav-
> devel/commit/b2f59861ee1a53c113fd37fe9378f739cc012042
>
> It also has implications for ignoring alerts from bytecode signatures that
> have
> VirusNames that aren't empty... I'll open a ticket for this
>
> Thanks!
>
> -Andrew
>
> On Mon, Mar 8, 2021 at 6:00 PM Mark Allan <markjal...@gmail.com> wrote:
>
> > Hi Andrew,
> >
> > Thanks for letting me know it's been dropped now. I was creating the
> > ign2 file almost identically, except for using double >> instead of
> > single as I already have dozens of lines in there.
> >
> > I see you have it without the .{} suffix. I tried both with it and
> > without and it wasn't working, ie
> > echo "BC.Img.Exploit.CVE_2018_4891-6453673-2" >> ignored.ign2
> > echo "BC.Img.Exploit.CVE_2018_4891-6453673-2.{}" >>
> > ignored.ign2
> >
> > Are you saying the .{} is no longer required to ignore bytecode signatures?
> >
> > Thanks again
> > Mark
> >
> > > On 8 Mar 2021, at 5:44 pm, Andrew Williams <awill...@sourcefire.com>
> > wrote:
> > >
> > > Thanks for reporting this Mark. The signature has been dropped and
> > > a new bytecode.cvd released.
> > >
> > > I was able to have the bytecode signature be ignored by creating the
> > .ign2
> > > file as follows and then moving it into the ClamAV signature directory:
> > > `echo "BC.Img.Exploit.CVE_2018_4891-6453673-2" > test.ign2`. Can
> > > you elaborate on how you are creating the .ign2 file?
> > >
> > > Thanks again,
> > >
> > > -Andrew
> > >
> > > On Thu, Mar 4, 2021 at 11:16 AM Mark Allan <markjal...@gmail.com>
> wrote:
> > >
> > >> Looks like we have another one!
> > >> BC.Img.Exploit.CVE_2018_4891-6453673-2
> > >>
> > >> This is generating loads of FPs as well.
> > >>
> > >> Curiously (and sorry for listing two issues in one email) adding a
> > >> bytecode signature name (with the .{} suffix) to an ign2 file
> > >> appears to have no effect. Any thoughts why this might be?
> > >>
> > >> Best regards,
> > >> Mark
> > >>
> > >>> On 16 Feb 2021, at 3:06 am, Micah Snyder (micasnyd) <
> > micas...@cisco.com>
> > >> wrote:
> > >>>
> > >>> It looks like BC.Img.Exploit.CVE_2017_11255-6335669-1 suffered the
> > >>> same
> > >> lack of proper FP testing as the other TIFF signature, likely for
> > >> the
> > same
> > >> reasons. After some time reviewing it, I agree that
> > >> BC.Img.Exploit.CVE_2017_11255-6335669-1 should be dropped. This
> > bytecode
> > >> signature has a relatively high probability to FP on TIFF files
> > >> that
> > don't
> > >> include a ColorMap in the IFD header(s), which is also fairly common.
> > >> Reworking the signature would is probably not worth the effort
> > considering
> > >> the CVE is from 2017.
> > >>>
> > >>> It should be dropped in the update tomorrow morning.
> > >>>
> > >>> Thanks for reaching out Mark.
> > >>>
> > >>> Regards,
> > >>> Micah
> > >>>
> > >>>> -----Original Message-----
> > >>>> From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On
> > >>>> Behalf
> > Of
> > >>>> Micah Snyder (micasnyd)
> > >>>> Sent: Monday, February 15, 2021 11:36 AM
> > >>>> To: ClamAV Development <clamav-devel@lists.clamav.net>
> > >>>> Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1
> > >>>>
> > >>>> Oh, sorry I misread your email. Needed more coffee. You were
> > >>>> asking
> > >> about
> > >>>> a different signature: BC.Img.Exploit.CVE_2017_11255-6335669-1
> > >>>> Will investigate.
> > >>>>
> > >>>> -Micah
> > >>>>
> > >>>>> -----Original Message-----
> > >>>>> From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On
> > >>>>> Behalf Of Micah Snyder (micasnyd)
> > >>>>> Sent: Monday, February 15, 2021 10:28 AM
> > >>>>> To: ClamAV Development <clamav-devel@lists.clamav.net>
> > >>>>> Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1
> > >>>>>
> > >>>>> Hi Mark,
> > >>>>>
> > >>>>> TL;DR: The type detection mismatch is fixed in the current
> > >>>>> daily +
> > >> 0.103.1.
> > >>>>> The issue was with the signature. We didn't know about it
> > >>>>> because of the mismatch. You should've found that the offending
> > >>>>> signature was dropped on Saturday morning.
> > >>>>>
> > >>>>> Details:
> > >>>>>
> > >>>>> 0.103.1 introduced CL_TYPE_TIFF and changed TIFF file type
> > recognition
> > >>>>> from:
> > >>>>> 0:0:49492a00:TIFF Little Endian:CL_TYPE_ANY:CL_TYPE_GRAPHICS
> > >>>>> 0:0:4d4d:TIFF Big Endian:CL_TYPE_ANY:CL_TYPE_ GRAPHICS
> > >>>>> to:
> > >>>>> 0:0:49492a00:TIFF Little Endian:CL_TYPE_ANY:CL_TYPE_TIFF
> > >>>>> 0:0:4d4d:TIFF Big Endian:CL_TYPE_ANY:CL_TYPE_TIFF
> > >>>>>
> > >>>>> When FTM signatures are loaded from daily.cvd, it overrides the
> > >>>>> built-in FTM signatures. So it turns out that daily's FTM file
> > >>>>> had been missing the original CL_TYPE_GRAPHICS detection of TIFF
> > >>>>> files
> > all
> > >>>>> this time, which would've been required for Target:5 signatures
> > >>>>> to alert on TIFF files. As a result, the signature in question
> > >>>>> "worked"
> > >>>>> in testing (with a single LDB file, using built-in FTM), but
> > >>>>> never worked in worked during FP testing or in production (with
> > >>>>> a daily CVD
> > >> file).
> > >>>>>
> > >>>>> When we added this to daily.ftm to support 0.103.1:
> > >>>>> 0:0:49492a00:TIFF Little Endian:CL_TYPE_ANY:CL_TYPE_TIFF:122
> > >>>>> 0:0:4d4d:TIFF Big Endian:CL_TYPE_ANY:CL_TYPE_TIFF:122
> > >>>>> ... all of a sudden a signature which was written for TIFF files
> > >>>>> started alerting on TIFF files (as it should've) because the new
> > >>>>> CL_TYPE_TIFF also alerts on
> > >>>>> Target:5 (graphics) types. We never added the CL_TYPE_GRAPHICS
> > >>>>> variant for 0.103.0 and prior, which is why it appeared to be an
> > issue
> > >> with
> > >>>> 0.103.1.
> > >>>>> Perhaps we should? I'll ask MRT about it.
> > >>>>>
> > >>>>> Anyways, this is basically a reminder that we need to make sure
> > >>>>> daily FTM and libclamav's FTM are in sync.
> > >>>>>
> > >>>>> -Micah
> > >>>>>
> > >>>>>
> > >>>>>> -----Original Message-----
> > >>>>>> From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On
> > Behalf
> > >>>>>> Of Mark Allan
> > >>>>>> Sent: Saturday, February 13, 2021 3:35 PM
> > >>>>>> To: ClamAV Development <clamav-devel@lists.clamav.net>
> > >>>>>> Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1
> > >>>>>>
> > >>>>>> Thanks. I've just found another one too
> > >>>>>>
> > >>>>>> BC.Img.Exploit.CVE_2017_11255-6335669-1
> > >>>>>>
> > >>>>>> It's triggering on a file that's been part of macOS for many years.
> > >>>>>> It's also a tiff file. I can submit this as well if necessary?
> > >>>>>>
> > >>>>>> Out of interest, is the type detection mismatch something that
> > >>>>>> can be fixed in daily.cvd or can I patch
> > >>>>>> libclamav/filetypes_int.h to revert it to what it was at 0.103.0?
> > >>>>>>
> > >>>>>> Mark
> > >>>>>>
> > >>>>>>> On 12 Feb 2021, at 5:23 am, Micah Snyder (micasnyd)
> > >>>>>> <micas...@cisco.com> wrote:
> > >>>>>>>
> > >>>>>>> It appears to me to be an issue with the signature which is
> > >>>>>>> only evident in
> > >>>>>> 0.103.1 now that we're matching TIFFs with Target:5 signatures,
> > >>>>>> like this
> > >>>>> one.
> > >>>>>>>
> > >>>>>>> There was apparently a mismatch for TIFF file type detection
> > >>>>>>> between the
> > >>>>>> file type magic signatures built-in to libclamav
> > >>>>>> (libclamav/filetypes_int.h) and the .ftm sigs shipped with
> > >>>>>> daily.cvd (which override the internal ones when loaded).
> > >>>>>>>
> > >>>>>>> I'll ask to have the signature dropped and re-evaluated.
> > >>>>>>>
> > >>>>>>> -Micah
> > >>>>>>>
> > >>>>>>>> -----Original Message-----
> > >>>>>>>> From: clamav-devel <clamav-devel-boun...@lists.clamav.net> On
> > >>>>>>>> Behalf Of Micah Snyder (micasnyd)
> > >>>>>>>> Sent: Thursday, February 11, 2021 8:27 PM
> > >>>>>>>> To: ClamAV Development <clamav-devel@lists.clamav.net>
> > >>>>>>>> Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1
> > >>>>>>>>
> > >>>>>>>> Thank you Mark! We'll take a look.
> > >>>>>>>>
> > >>>>>>>> -Micah
> > >>>>>>>>
> > >>>>>>>>> -----Original Message-----
> > >>>>>>>>> From: clamav-devel <clamav-devel-boun...@lists.clamav.net>
> > >>>>>>>>> On
> > >>>>>> Behalf
> > >>>>>>>>> Of Mark Allan
> > >>>>>>>>> Sent: Thursday, February 11, 2021 3:54 PM
> > >>>>>>>>> To: ClamAV Development <clamav-devel@lists.clamav.net>
> > >>>>>>>>> Subject: Re: [Clamav-devel] Issue with FP only on 0.103.1
> > >>>>>>>>>
> > >>>>>>>>> Hi Micah,
> > >>>>>>>>>
> > >>>>>>>>> Yes of course! I've just uploaded a zip file (Archive.zip)
> > >>>>>>>>> to the FP page on clamav.net
> > >>>>>>>>> MD5 (Archive.zip) = 45229d954a884a1e03aba15b9f42168a
> > >>>>>>>>>
> > >>>>>>>>> Regards
> > >>>>>>>>> Mark
> > >>>>>>>>>
> > >>>>>>>>>> On 11 Feb 2021, at 7:12 pm, Micah Snyder (micasnyd)
> > >>>>>>>>> <micas...@cisco.com> wrote:
> > >>>>>>>>>>
> > >>>>>>>>>> Hi Mark,
> > >>>>>>>>>>
> > >>>>>>>>>> Do you think you could share a sample or two with me to test.
> > >>>>>>>>>> I'm really
> > >>>>>>>>> curious what changed and would like to debug each version
> > >>>>>>>>> with a sample or two.
> > >>>>>>>>>>
> > >>>>>>>>>> -Micah
> > >>>>>>>>>>
> > >>>>>>>>>>> -----Original Message-----
> > >>>>>>>>>>> From: clamav-devel <clamav-devel-boun...@lists.clamav.net>
> > >>>>>>>>>>> On Behalf Of Mark Allan
> > >>>>>>>>>>> Sent: Monday, February 8, 2021 3:04 AM
> > >>>>>>>>>>> To: ClamAV Development <clamav-devel@lists.clamav.net>
> > >>>>>>>>>>> Subject: [Clamav-devel] Issue with FP only on 0.103.1
> > >>>>>>>>>>>
> > >>>>>>>>>>> Hi all,
> > >>>>>>>>>>>
> > >>>>>>>>>>> It looks like the additional image file type support in
> > >>>>>>>>>>> 0.103.1 has introduced an issue with a particular
> > >>>>>>>>>>> signature which has been in the database since 2018
> > >>>>>>>>>>>
> > >>>>>>>>>>> Img.Exploit.CVE_2018_4904-6449838-0
> > >>>>>>>>>>>
> > >>>>>>>>>>> It's flagging up thousands of known-good files. As far as
> > >>>>>>>>>>> I can tell, they're all TIFF files.
> > >>>>>>>>>>>
> > >>>>>>>>>>> I've added that signature to an ign2 file for now, but I'm
> > >>>>>>>>>>> wondering if there's something else that's maybe amiss
> > >>>>>>>>>>> somewhere either with the signature or the 0.103.1 update?
> > >>>>>>>>>>>
> > >>>>>>>>>>> Best regards,
> > >>>>>>>>>>> Mark
> > >>>>>>>>>>>
> > >>>>>>>>>>> _______________________________________________
> > >>>>>>>>>>>
> > >>>>>>>>>>> clamav-devel mailing list
> > >>>>>>>>>>> clamav-devel@lists.clamav.net
> > >>>>>>>>>>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>>>>>>>>>
> > >>>>>>>>>>> Please submit your patches to our Github:
> > >>>>>>>>>>> https://github.com/Cisco- Talos/clamav-devel/pulls
> > >>>>>>>>>>>
> > >>>>>>>>>>> Help us build a comprehensive ClamAV guide:
> > >>>>>>>>>>> https://github.com/vrtadmin/clamav-faq
> > >>>>>>>>>>>
> > >>>>>>>>>>> http://www.clamav.net/contact.html#ml
> > >>>>>>>>>> _______________________________________________
> > >>>>>>>>>>
> > >>>>>>>>>> clamav-devel mailing list
> > >>>>>>>>>> clamav-devel@lists.clamav.net
> > >>>>>>>>>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>>>>>>>>
> > >>>>>>>>>> Please submit your patches to our Github:
> > >>>>>>>>>> https://github.com/Cisco-Talos/clamav-devel/pulls
> > >>>>>>>>>>
> > >>>>>>>>>> Help us build a comprehensive ClamAV guide:
> > >>>>>>>>>> https://github.com/vrtadmin/clamav-faq
> > >>>>>>>>>>
> > >>>>>>>>>> http://www.clamav.net/contact.html#ml
> > >>>>>>>>>
> > >>>>>>>>> _______________________________________________
> > >>>>>>>>>
> > >>>>>>>>> clamav-devel mailing list
> > >>>>>>>>> clamav-devel@lists.clamav.net
> > >>>>>>>>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>>>>>>>
> > >>>>>>>>> Please submit your patches to our Github:
> > >>>>>>>>> https://github.com/Cisco- Talos/clamav-devel/pulls
> > >>>>>>>>>
> > >>>>>>>>> Help us build a comprehensive ClamAV guide:
> > >>>>>>>>> https://github.com/vrtadmin/clamav-faq
> > >>>>>>>>>
> > >>>>>>>>> http://www.clamav.net/contact.html#ml
> > >>>>>>>> _______________________________________________
> > >>>>>>>>
> > >>>>>>>> clamav-devel mailing list
> > >>>>>>>> clamav-devel@lists.clamav.net
> > >>>>>>>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>>>>>>
> > >>>>>>>> Please submit your patches to our Github:
> > >>>>>>>> https://github.com/Cisco- Talos/clamav-devel/pulls
> > >>>>>>>>
> > >>>>>>>> Help us build a comprehensive ClamAV guide:
> > >>>>>>>> https://github.com/vrtadmin/clamav-faq
> > >>>>>>>>
> > >>>>>>>> http://www.clamav.net/contact.html#ml
> > >>>>>>> _______________________________________________
> > >>>>>>>
> > >>>>>>> clamav-devel mailing list
> > >>>>>>> clamav-devel@lists.clamav.net
> > >>>>>>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>>>>>
> > >>>>>>> Please submit your patches to our Github:
> > >>>>>>> https://github.com/Cisco-Talos/clamav-devel/pulls
> > >>>>>>>
> > >>>>>>> Help us build a comprehensive ClamAV guide:
> > >>>>>>> https://github.com/vrtadmin/clamav-faq
> > >>>>>>>
> > >>>>>>> http://www.clamav.net/contact.html#ml
> > >>>>>>
> > >>>>>> _______________________________________________
> > >>>>>>
> > >>>>>> clamav-devel mailing list
> > >>>>>> clamav-devel@lists.clamav.net
> > >>>>>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>>>>
> > >>>>>> Please submit your patches to our Github:
> > >>>>>> https://github.com/Cisco- Talos/clamav-devel/pulls
> > >>>>>>
> > >>>>>> Help us build a comprehensive ClamAV guide:
> > >>>>>> https://github.com/vrtadmin/clamav-faq
> > >>>>>>
> > >>>>>> http://www.clamav.net/contact.html#ml
> > >>>>> _______________________________________________
> > >>>>>
> > >>>>> clamav-devel mailing list
> > >>>>> clamav-devel@lists.clamav.net
> > >>>>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>>>
> > >>>>> Please submit your patches to our Github:
> > >>>>> https://github.com/Cisco- Talos/clamav-devel/pulls
> > >>>>>
> > >>>>> Help us build a comprehensive ClamAV guide:
> > >>>>> https://github.com/vrtadmin/clamav-faq
> > >>>>>
> > >>>>> http://www.clamav.net/contact.html#ml
> > >>>> _______________________________________________
> > >>>>
> > >>>> clamav-devel mailing list
> > >>>> clamav-devel@lists.clamav.net
> > >>>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>>
> > >>>> Please submit your patches to our Github:
> > >>>> https://github.com/Cisco- Talos/clamav-devel/pulls
> > >>>>
> > >>>> Help us build a comprehensive ClamAV guide:
> > >>>> https://github.com/vrtadmin/clamav-faq
> > >>>>
> > >>>> http://www.clamav.net/contact.html#ml
> > >>> _______________________________________________
> > >>>
> > >>> clamav-devel mailing list
> > >>> clamav-devel@lists.clamav.net
> > >>> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>>
> > >>> Please submit your patches to our Github:
> > >> https://github.com/Cisco-Talos/clamav-devel/pulls
> > >>>
> > >>> Help us build a comprehensive ClamAV guide:
> > >>> https://github.com/vrtadmin/clamav-faq
> > >>>
> > >>> http://www.clamav.net/contact.html#ml
> > >>
> > >> _______________________________________________
> > >>
> > >> clamav-devel mailing list
> > >> clamav-devel@lists.clamav.net
> > >> https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >>
> > >> Please submit your patches to our Github:
> > >> https://github.com/Cisco-Talos/clamav-devel/pulls
> > >>
> > >> Help us build a comprehensive ClamAV guide:
> > >> https://github.com/vrtadmin/clamav-faq
> > >>
> > >> http://www.clamav.net/contact.html#ml
> > >>
> > > _______________________________________________
> > >
> > > clamav-devel mailing list
> > > clamav-devel@lists.clamav.net
> > > https://lists.clamav.net/mailman/listinfo/clamav-devel
> > >
> > > Please submit your patches to our Github:
> > https://github.com/Cisco-Talos/clamav-devel/pulls
> > >
> > > Help us build a comprehensive ClamAV guide:
> > > https://github.com/vrtadmin/clamav-faq
> > >
> > > http://www.clamav.net/contact.html#ml
> >
> > _______________________________________________
> >
> > clamav-devel mailing list
> > clamav-devel@lists.clamav.net
> > https://lists.clamav.net/mailman/listinfo/clamav-devel
> >
> > Please submit your patches to our Github:
> > https://github.com/Cisco-Talos/clamav-devel/pulls
> >
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/vrtadmin/clamav-faq
> >
> > http://www.clamav.net/contact.html#ml
> >
> _______________________________________________
>
> clamav-devel mailing list
> clamav-devel@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-devel
>
> Please submit your patches to our Github: https://github.com/Cisco-
> Talos/clamav-devel/pulls
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
_______________________________________________
clamav-devel mailing list
clamav-devel@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-devel
Please submit your patches to our Github:
https://github.com/Cisco-Talos/clamav-devel/pulls
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
