Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned
If ClamAV always rounded up when counting the number of 16kb blocks, then it should be counting at least 0.016384 MB (or 0.015625 MiB) for tiny files. By normal rounding rules this should display as 0.02 MB/MiB. On Tue, 3 Nov 2020 17:50:18 + Mark Fortescue via clamav-users wrote: > Hi all, > > I would call this a bug. Scanning 1 byte is the same as scanning 1 block. > > When storing things in blocks is is always important to round up or you > get a false impression of reality. > > You can't store 100 bytes in 0 disk sectors of 128 bytes. It is always 1 > disk sector. > > Can you not just round up by adding (BlockSize - 1) bytes when setting > the block variables ? > > Regards > Mark. > > On 03/11/2020 16:07, Paul Kosinski via clamav-users wrote: > > "This is a display problem, not a storage problem." > > > > I disagree. When the counts in info.blocks and info.rblocks are counts > > of 16kb *blocks*, keeping precise track of the reading and scanning of > > small files is impossible, no matter how clever the display code is. > > > > > > > > On Tue, 3 Nov 2020 17:44:18 +1100 > > "Gary R. Schmidt" wrote: > > > >> On 03/11/2020 16:00, Paul Kosinski via clamav-users wrote: > >>> "(don't you love C?)" > >>> > >>> I have never understood why the originators of C didn't give integers > >>> explicit widths in bits: their scheme made C code often non-portable. > >>> > >> Because C is intended to be very, very close to the machine > >> architecture, only a step or tow above assembler, or doing the > >> bit-twiddling by hand. > >> > >>> When I wrote code in the mid 1990s for the DEC Alpha, ints were 32 bits > >>> while longs were 64 (unlike "standard" C). This made Alpha C code not > >>> portable to lesser CPUs. On the other hand, when I wrote C on DOS for > >>> the IBM PC in the late 1980s, ints were only 8 bits! It took some time > >>> to figure out why my C-compliant code failed so badly. In spite of all > >>> that, having started programming before C was invented, I can safely > >>> say that C is better than its predecessors for software like ClamAV. > >>> > >> Uh, not a good example, I've written C code that is still in use on > >> everything from 80286s (yes, Virginia, there are people who keep them > >> alive, not just because they're cheap, sometimes just because they > >> *can*) to DEC Alphas and Power and SPARC64 and PA-RISC, it's just a > >> matter of knowing what you are doing, and sticking to it... > >> > >>> P.S. Good code these days tends to use typedefs defining things like > >>> int32, uint64 etc. A shame the original ClamAV coders didn't do that. > >>> > >> And none of this has *anything* to do with the original problem - seeing > >> 0 when the value is 0.01, or so. > >> > >> This is a display problem, not a storage problem. You could declare > >> something as PIC(999.99) and you will still only see 0 > >> if you told it to display two decimal places. > >> > >>Cheers, > >>GaryB-) > ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned
Thanks a lot for all your inputs.
Further during the test I tried to scan a ".mp3" file of size 5.04MB. The
returned log mentions the number of Scanned files as 1, but when I see the
Data Scanned - it is 0.00 MB. Does ClamAV support scanning MP3 files? Has
anyone tried scanning MP3 files?
{'/tmp/bucket-file-upload/2copy.mp3': 'OK', 'Known viruses': '8931107',
'Engine version': '0.102.4', 'Scanned directories': '0', 'Scanned files':
'1', 'Infected files': '0', 'Data scanned': '0.00 MB', 'Data read': '5.04
MB (ratio 0.00:1)', 'Time': '22.727 sec (0 m 22 s)'}
I am using "clamscan" inside a Lambda function to scan the file which is
uploaded to an AWS S3 bucket.
regards
Ankur
On Wed, Nov 4, 2020 at 4:51 AM Mark Fortescue via clamav-users <
clamav-users@lists.clamav.net> wrote:
> Hi all,
>
> I would call this a bug. Scanning 1 byte is the same as scanning 1 block.
>
> When storing things in blocks is is always important to round up or you
> get a false impression of reality.
>
> You can't store 100 bytes in 0 disk sectors of 128 bytes. It is always 1
> disk sector.
>
> Can you not just round up by adding (BlockSize - 1) bytes when setting
> the block variables ?
>
> Regards
> Mark.
>
> On 03/11/2020 16:07, Paul Kosinski via clamav-users wrote:
> > "This is a display problem, not a storage problem."
> >
> > I disagree. When the counts in info.blocks and info.rblocks are counts
> > of 16kb *blocks*, keeping precise track of the reading and scanning of
> > small files is impossible, no matter how clever the display code is.
> >
> >
> >
> > On Tue, 3 Nov 2020 17:44:18 +1100
> > "Gary R. Schmidt" wrote:
> >
> >> On 03/11/2020 16:00, Paul Kosinski via clamav-users wrote:
> >>> "(don't you love C?)"
> >>>
> >>> I have never understood why the originators of C didn't give integers
> >>> explicit widths in bits: their scheme made C code often non-portable.
> >>>
> >> Because C is intended to be very, very close to the machine
> >> architecture, only a step or tow above assembler, or doing the
> >> bit-twiddling by hand.
> >>
> >>> When I wrote code in the mid 1990s for the DEC Alpha, ints were 32 bits
> >>> while longs were 64 (unlike "standard" C). This made Alpha C code not
> >>> portable to lesser CPUs. On the other hand, when I wrote C on DOS for
> >>> the IBM PC in the late 1980s, ints were only 8 bits! It took some time
> >>> to figure out why my C-compliant code failed so badly. In spite of all
> >>> that, having started programming before C was invented, I can safely
> >>> say that C is better than its predecessors for software like ClamAV.
> >>>
> >> Uh, not a good example, I've written C code that is still in use on
> >> everything from 80286s (yes, Virginia, there are people who keep them
> >> alive, not just because they're cheap, sometimes just because they
> >> *can*) to DEC Alphas and Power and SPARC64 and PA-RISC, it's just a
> >> matter of knowing what you are doing, and sticking to it...
> >>
> >>> P.S. Good code these days tends to use typedefs defining things like
> >>> int32, uint64 etc. A shame the original ClamAV coders didn't do that.
> >>>
> >> And none of this has *anything* to do with the original problem - seeing
> >> 0 when the value is 0.01, or so.
> >>
> >> This is a display problem, not a storage problem. You could declare
> >> something as PIC(999.99) and you will still only see 0
> >> if you told it to display two decimal places.
> >>
> >> Cheers,
> >> GaryB-)
> >
> >
> > ___
> >
> > clamav-users mailing list
> > clamav-users@lists.clamav.net
> > https://lists.clamav.net/mailman/listinfo/clamav-users
> >
> >
> > Help us build a comprehensive ClamAV guide:
> > https://github.com/vrtadmin/clamav-faq
> >
> > http://www.clamav.net/contact.html#ml
> >
>
> ___
>
> clamav-users mailing list
> clamav-users@lists.clamav.net
> https://lists.clamav.net/mailman/listinfo/clamav-users
>
>
> Help us build a comprehensive ClamAV guide:
> https://github.com/vrtadmin/clamav-faq
>
> http://www.clamav.net/contact.html#ml
>
--
regards
Ankur
+61481141085
___
clamav-users mailing list
clamav-users@lists.clamav.net
https://lists.clamav.net/mailman/listinfo/clamav-users
Help us build a comprehensive ClamAV guide:
https://github.com/vrtadmin/clamav-faq
http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned
Hi all, I would call this a bug. Scanning 1 byte is the same as scanning 1 block. When storing things in blocks is is always important to round up or you get a false impression of reality. You can't store 100 bytes in 0 disk sectors of 128 bytes. It is always 1 disk sector. Can you not just round up by adding (BlockSize - 1) bytes when setting the block variables ? Regards Mark. On 03/11/2020 16:07, Paul Kosinski via clamav-users wrote: "This is a display problem, not a storage problem." I disagree. When the counts in info.blocks and info.rblocks are counts of 16kb *blocks*, keeping precise track of the reading and scanning of small files is impossible, no matter how clever the display code is. On Tue, 3 Nov 2020 17:44:18 +1100 "Gary R. Schmidt" wrote: On 03/11/2020 16:00, Paul Kosinski via clamav-users wrote: "(don't you love C?)" I have never understood why the originators of C didn't give integers explicit widths in bits: their scheme made C code often non-portable. Because C is intended to be very, very close to the machine architecture, only a step or tow above assembler, or doing the bit-twiddling by hand. When I wrote code in the mid 1990s for the DEC Alpha, ints were 32 bits while longs were 64 (unlike "standard" C). This made Alpha C code not portable to lesser CPUs. On the other hand, when I wrote C on DOS for the IBM PC in the late 1980s, ints were only 8 bits! It took some time to figure out why my C-compliant code failed so badly. In spite of all that, having started programming before C was invented, I can safely say that C is better than its predecessors for software like ClamAV. Uh, not a good example, I've written C code that is still in use on everything from 80286s (yes, Virginia, there are people who keep them alive, not just because they're cheap, sometimes just because they *can*) to DEC Alphas and Power and SPARC64 and PA-RISC, it's just a matter of knowing what you are doing, and sticking to it... P.S. Good code these days tends to use typedefs defining things like int32, uint64 etc. A shame the original ClamAV coders didn't do that. And none of this has *anything* to do with the original problem - seeing 0 when the value is 0.01, or so. This is a display problem, not a storage problem. You could declare something as PIC(999.99) and you will still only see 0 if you told it to display two decimal places. Cheers, GaryB-) ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] ClamAV Scan - Data Read vs Data Scanned
"This is a display problem, not a storage problem." I disagree. When the counts in info.blocks and info.rblocks are counts of 16kb *blocks*, keeping precise track of the reading and scanning of small files is impossible, no matter how clever the display code is. On Tue, 3 Nov 2020 17:44:18 +1100 "Gary R. Schmidt" wrote: > On 03/11/2020 16:00, Paul Kosinski via clamav-users wrote: > > "(don't you love C?)" > > > > I have never understood why the originators of C didn't give integers > > explicit widths in bits: their scheme made C code often non-portable. > > > Because C is intended to be very, very close to the machine > architecture, only a step or tow above assembler, or doing the > bit-twiddling by hand. > > > When I wrote code in the mid 1990s for the DEC Alpha, ints were 32 bits > > while longs were 64 (unlike "standard" C). This made Alpha C code not > > portable to lesser CPUs. On the other hand, when I wrote C on DOS for > > the IBM PC in the late 1980s, ints were only 8 bits! It took some time > > to figure out why my C-compliant code failed so badly. In spite of all > > that, having started programming before C was invented, I can safely > > say that C is better than its predecessors for software like ClamAV. > > > Uh, not a good example, I've written C code that is still in use on > everything from 80286s (yes, Virginia, there are people who keep them > alive, not just because they're cheap, sometimes just because they > *can*) to DEC Alphas and Power and SPARC64 and PA-RISC, it's just a > matter of knowing what you are doing, and sticking to it... > > > P.S. Good code these days tends to use typedefs defining things like > > int32, uint64 etc. A shame the original ClamAV coders didn't do that. > > > And none of this has *anything* to do with the original problem - seeing > 0 when the value is 0.01, or so. > > This is a display problem, not a storage problem. You could declare > something as PIC(999.99) and you will still only see 0 > if you told it to display two decimal places. > > Cheers, > GaryB-) ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
Re: [clamav-users] Errir parsing PNG files and 451_mail_server_temporarily_rejected_message
The gif I sent is part of teh signature of a mail Nothing important On 11/2/2020 11:04 PM, Micah Snyder (micasnyd) via clamav-users wrote: Thanks Pablo, Just took a look - it seems that image001.gif is missing the final byte, a value "0x3B" should be at the end of every GIF file. Interestingly firefox rendered it, though clam didn't like it and 010editor's GIF template also couldn't handle it. Perhaps this is the sort of issue that should be overlooked when validating the format? I wonder if this is a common occurrence. I haven't seen it in my test set yet, but I'm still collecting more files to test with. -Micah On 11/2/20, 5:58 PM, "clamav-users on behalf of Pablo Murillo" wrote: I try so send several mails to the list with gifs attached, but none arrived to the list I'm sending a link to one gif that everyday is rejected by ClamAV Link to gif file: http://pablomurillo.com.ar/image001.gif On 11/2/2020 9:02 PM, Micah Snyder (micasnyd) via clamav-users wrote: > We disabled the PNG parser because of bugs in the code that need to be fixed. I'm actively working on the PNG and GIF parsers, and have started testing with larger sample sets to try to make sure they're more robust. T > > he GIF parser seems fairly stable though I'm putting a lot of effort into making the code readable and adding debug output. If you have GIF samples that are incorrectly throwing parse errors, can you share them with me? It would be very helpful. > > -Micah > > On 11/2/20, 11:50 AM, "clamav-users on behalf of Pablo Murillo" wrote: > > Hi > > What happened with this ? > Now I'm seeing some errors in GIF files too > > On 10/21/2020 2:04 PM, Pablo Murillo wrote: > > Ajajaja > > I feel better ! > > ajajaja > > > > Check the mail I sent few minutes ago with more info and more files > > (defug, config, the rights pngs, and the real error) > > > > Sorry my english ! > > > > > > On 10/21/2020 1:48 PM, Micah Snyder (micasnyd) via clamav-users wrote: > >> Thanks Pablo I'll check it out right away! I hope you're feeling > >> better now :D > >> > >> Best, > >> Micah > >> > >> On 10/20/20, 1:10 PM, "clamav-users on behalf of Pablo Murillo" > >> >> i...@pablomurillo.com.ar> wrote: > >> > >> Hi Micah > >> > >> I was ready to send a new mail when yours arrived > >> I made a lot of test and when I activated the option LogClean to > >> yes on > >> clamd.conf I found that not all PNG generate the problem ! > >> I'm sending some PNGs attached inside a tar.gz > >> > >> Now, I feel me better, I thought that I was the problem :D > >> > >> Pablo Murillo > >> > >> On 10/20/2020 4:40 PM, Micah Snyder (micasnyd) via clamav-users > >> wrote: > >> > Hi all, > >> > > >> > It seems as though the new PNG graphics format/CVE checker > >> added in 0.103 is causing trouble for you and for some others. We > >> will disable it for now, which we can do with an update to the daily > >> database. > >> > > >> > Pablo, if you're allowed to share some of the PNG files with > >> me that caused issues for you, the samples will help us find the bug > >> in the PNG parser. > >> > > >> > Regards, > >> > Micah > >> > > >> > > >> > Micah Snyder > >> > ClamAV Development > >> > Talos > >> > Cisco Systems, Inc. > >> > > >> > > >> > > >> > On 10/19/20, 11:26 AM, "clamav-users on behalf of Pablo > >> Murillo" >> i...@pablomurillo.com.ar> wrote: > >> > > >> > Hi > >> > > >> > I started seeing the error [ > >> > 451_mail_server_temporarily_rejected_message ] on > >> maillog, and looking > >> > in the clamd.log I found a lot of lines like this [ > >> /*/*/*/*.png: Can't > >> > parse data ERROR ] > >> > I added on clamd.conf the next line > >> > > >> > ExcludePath ^\.png$ > >> > > >> > But clamd is still sending the error > >> > Is there a way to avoid this ? > >> > Any mail qith a PNG file is rejected by clamd > >> > > >> >
Re: [clamav-users] Errir parsing PNG files and 451_mail_server_temporarily_rejected_message
Hi there, On Tue, 3 Nov 2020, Micah Snyder (micasnyd) via clamav-users wrote: Just took a look - it seems that image001.gif is missing the final byte, a value "0x3B" should be at the end of every GIF file. ... I wonder if this is a common occurrence. ... It seems to be common enough to need to be able to cope with it. I just ran 'locate' on my laptop to find every .gif file on the box, piped the output through 'tail -c1' and 'hexdump -f something' to get the final byte of each file in hex, through a 'perl -ap' one-liner to count the distinct final bytes, and finally through 'sort'. Results from 5970 GIF files: 01 1 3e 1 ad 1 82 7 0a 8 d9 32 00 45 3b 5875 I've almost no idea where all these files came from. Many will have been parts of packages, many will be just interesting images that I grabbed from articles I've read over the years. It's possibile that at least some of them are broken bits from failed browser processes etc. I spent no time looking for clues. Many of the files are very old. The oldest file is 26 years old - as it happens, to the day! Please let me know if I can do any more with this. -- 73, Ged. ___ clamav-users mailing list clamav-users@lists.clamav.net https://lists.clamav.net/mailman/listinfo/clamav-users Help us build a comprehensive ClamAV guide: https://github.com/vrtadmin/clamav-faq http://www.clamav.net/contact.html#ml
