Re: Rules for invisible div and 0pt font?

[Amir Caspi]

On Jun 18, 2019, at 2:21 AM, Giovanni Bechis  wrote:
> 
>> rawbody  AC_HIDDEN_FONT  /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
>> 
> There is T_HIDDEN_WORD on my sandbox 
> (https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
> I have just committed a more generic version.

Unfortunately I'm still seeing a bunch of spams with font-size: 0px that aren't 
hitting any sort of "hidden font" or "tiny font" rule.

The above suggested rule would catch those, in case someone can try sandboxing 
that.  I had also suggested matching on line-height:0 and similar, but it 
appears that those might be used in hams as well... so we might want to limit 
it to just font-size.

But it looks like Giovanni's T_GB_HIDDEN_WORD isn't scoring so well lately... 
not sure how it compares to my suggestion above.

Cheers.

--- Amir

Re: Rules for invisible div and 0pt font?

[Paul Stead]

Just going from
http://svn.apache.org/viewvc/spamassassin/trunk/rulesrc/sandbox/gbechis/20_html.cf?revision=1861560

Re: Rules for invisible div and 0pt font?

[John Hardin]

On Tue, 18 Jun 2019, Paul Stead wrote:


On Tue, 18 Jun 2019 at 19:14, John Hardin  wrote:


On Tue, 18 Jun 2019, Giovanni Bechis wrote:


On 6/17/19 9:14 PM, Amir Caspi wrote:

There is a div here with display:none, as well as font-size:0px.  The

spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule
relating to a hidden div or tiny font.


There is T_HIDDEN_WORD on my sandbox (

https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)

I have just committed a more generic version.


You probably also want to add "tflags publish" if its performance is
acceptable to you.



Also rename from T_ otherwise it will be skipped. If you drop the T_ and
omit the publish it will let QA decide if performance is good enough :)


That's only if you explicitly named it with the T_ prefix. That prefix is 
automatically added in some cases.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  All I could think about was this bear is so close to me I can
  see its teeth. I could have kissed it. I wished I had a gun.
 -- Alyson Jones-Robinson
---
 Today: SWMBO's Birthday

Re: Rules for invisible div and 0pt font?

[Paul Stead]

On Tue, 18 Jun 2019 at 20:23, Paul Stead  wrote:

> Also rename from T_ otherwise it will be skipped. If you drop the T_ and
> omit the publish it will let QA decide if performance is good enough :)
>

Although not looking so good today -
https://ruleqa.spamassassin.org/20190618-r1861562-n/T_HIDDEN_WORD/detail

Paul

Re: Rules for invisible div and 0pt font?

[Paul Stead]

On Tue, 18 Jun 2019 at 19:14, John Hardin  wrote:

> On Tue, 18 Jun 2019, Giovanni Bechis wrote:
>
> > On 6/17/19 9:14 PM, Amir Caspi wrote:
> >> There is a div here with display:none, as well as font-size:0px.  The
> spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule
> relating to a hidden div or tiny font.
> >
> > There is T_HIDDEN_WORD on my sandbox (
> https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
> > I have just committed a more generic version.
>
> You probably also want to add "tflags publish" if its performance is
> acceptable to you.
>

Also rename from T_ otherwise it will be skipped. If you drop the T_ and
omit the publish it will let QA decide if performance is good enough :)


Paul

Re: Rules for invisible div and 0pt font?

[John Hardin]

On Tue, 18 Jun 2019, Giovanni Bechis wrote:


On 6/17/19 9:14 PM, Amir Caspi wrote:

There is a div here with display:none, as well as font-size:0px.  The spample 
hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule relating to a 
hidden div or tiny font.


There is T_HIDDEN_WORD on my sandbox 
(https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
I have just committed a more generic version.


You probably also want to add "tflags publish" if its performance is 
acceptable to you.


--
 John Hardin KA7OHZhttp://www.impsec.org/~jhardin/
 jhar...@impsec.orgFALaholic #11174 pgpk -a jhar...@impsec.org
 key: 0xB8732E79 -- 2D8C 34F4 6411 F507 136C  AF76 D822 E6E6 B873 2E79
---
  Your mouse has moved. Your Windows Operating System must be
  relicensed due to this hardware change. Please contact Microsoft
  to obtain a new activation key. If this hardware change results in
  added functionality you may be subject to additional license fees.
  Your system will now shut down. Thank you for choosing Microsoft.
---
 Today: SWMBO's Birthday

Re: Rules for invisible div and 0pt font?

[Amir Caspi]

On Jun 18, 2019, at 10:55 AM, Bill Cole 
 wrote:
> 
> Looking at the 2 most recent (a USPS "Informed Delivery Daily Digest" message 
> and Office Depot order followup) I see display:none only in inline style 
> attributes of block elements.   e.g.:

Looks like the first one is a web bug.  The second one is more problematic, 
because it also includes things like width:0 and max-height:0, which the OTHER 
rule is intended to catch.

Ugh.

I guess those rules are likely to hit a lot of ham, but maybe there are some 
good metas...

--- Amir

Re: Rules for invisible div and 0pt font?

[Bill Cole]

On 18 Jun 2019, at 10:52, Amir Caspi wrote:

Are the matches all within @media blocks like lbutlr suggested or do 
they occur inline within div/span/etc as well?


Looking at the 2 most recent (a USPS "Informed Delivery Daily Digest" 
message and Office Depot order followup) I see display:none only in 
inline style attributes of block elements.   e.g.:





src=3D"http://pixel.watch/REDACT; styl=

e=3D"display:none;visibility:hidden" />

And:











Thanks!

--- Amir
thumbed via iPhone

On Jun 18, 2019, at 8:42 AM, Bill Cole 
 wrote:



On 17 Jun 2019, at 15:25, @lbutlr wrote:


On Jun 17, 2019, at 1:14 PM, Amir Caspi  wrote:
rawbodyAC_HIDDEN_ELEMENT/display\s*:\s*none\s*;/


Since display:none is a pretty common method for showing and hiding 
elements depending on things like screen size, I would guess this is 
going to hit mostly ham.


Mail in my personal recent archives matching that includes ham from 
USPS, Apple, Home Depot, Office Depot, Paypal, Fidelity, Subway, 
Kroger, and others, all of it non-bulk requested and expected mail. 
In short: valuable business to consumer transactional and/or 
account-specific mail.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)

Re: Rules for invisible div and 0pt font?

[Amir Caspi]

Are the matches all within @media blocks like lbutlr suggested or do they occur 
inline within div/span/etc as well?

Thanks!

--- Amir
thumbed via iPhone

> On Jun 18, 2019, at 8:42 AM, Bill Cole 
>  wrote:
> 
>> On 17 Jun 2019, at 15:25, @lbutlr wrote:
>> 
>>> On Jun 17, 2019, at 1:14 PM, Amir Caspi  wrote:
>>> rawbodyAC_HIDDEN_ELEMENT/display\s*:\s*none\s*;/
>> 
>> Since display:none is a pretty common method for showing and hiding elements 
>> depending on things like screen size, I would guess this is going to hit 
>> mostly ham.
> 
> Mail in my personal recent archives matching that includes ham from USPS, 
> Apple, Home Depot, Office Depot, Paypal, Fidelity, Subway, Kroger, and 
> others, all of it non-bulk requested and expected mail. In short: valuable 
> business to consumer transactional and/or account-specific mail.
> 
> --
> Bill Cole
> b...@scconsult.com or billc...@apache.org
> (AKA @grumpybozo and many *@billmail.scconsult.com addresses)

Re: Rules for invisible div and 0pt font?

[Bill Cole]

On 17 Jun 2019, at 15:25, @lbutlr wrote:


On Jun 17, 2019, at 1:14 PM, Amir Caspi  wrote:

rawbody AC_HIDDEN_ELEMENT   /display\s*:\s*none\s*;/


Since display:none is a pretty common method for showing and hiding 
elements depending on things like screen size, I would guess this is 
going to hit mostly ham.


Mail in my personal recent archives matching that includes ham from 
USPS, Apple, Home Depot, Office Depot, Paypal, Fidelity, Subway, Kroger, 
and others, all of it non-bulk requested and expected mail. In short: 
valuable business to consumer transactional and/or account-specific 
mail.


--
Bill Cole
b...@scconsult.com or billc...@apache.org
(AKA @grumpybozo and many *@billmail.scconsult.com addresses)

Re: Rules for invisible div and 0pt font?

[Dan Malm]

On 2019-06-17 21:26, Amir Caspi wrote:> On Jun 17, 2019, at 1:14 PM,
Amir Caspi  > wrote:
>>
>> rawbodyAC_HIDDEN_FONT/font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
>>
>
> Actually, based on another spample (https://pastebin.com/rrU2AsVT),
> let's modify this one -- the em/pt/px/% isn't required:
>
> rawbodyAC_HIDDEN_FONT/font-size\s*:\s*0\s*(?:em|pt|px|%)?\s*;/
>
> It might also be prudent to look for 0-height or 0-width line-height,
> max-height, max-width... so that would change the hidden-font to:
>
>
rawbodyAC_HIDDEN_FONT/(?:font-size|line-height|max-height|max-width)\s*:\s*0\s*(?:em|pt|px|%)?\s*;/
>
> And, looks like another rule might be useful:
>
> rawbodyAC_LARGE_NEG_INDENT/text-indent\s*:\s*-[0-9]{3,}(?:em|pt|px|%)\s*;/
>
> This looks for a large negative text-indent, as is used in the spample
> linked above.
>
> Cheers.
>
> --- Amir
>

Don't forget that css also has an "!important" flag that comes before
the semicolon.

rawbodyAC_HIDDEN_FONT/(?:font-size|line-height|max-height|max-width)\s*:\s*0\s*(?:em|pt|px|%)?(?:\s*!important)?\s*;/

BR/Mvh. Dan Malm, Systems Engineer, One.com



signature.asc
Description: OpenPGP digital signature

Re: Rules for invisible div and 0pt font?

[Giovanni Bechis]

On 6/17/19 9:14 PM, Amir Caspi wrote:
> Hi all,
> 
> In reviewing today's FNs I came across the following spample:
> https://pastebin.com/9QQVwUY6
> 
> There is a div here with display:none, as well as font-size:0px.  The spample 
> hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule relating to a 
> hidden div or tiny font.
> 
> Does LOW_CONTRAST include font-size too small, or just color too light?  Is 
> there a rule for matching display:none?
> 
> If not, may I propose that the following rules be sandboxed?
> 
> rawbody   AC_HIDDEN_ELEMENT   /display\s*:\s*none\s*;/
> 
> rawbody   AC_HIDDEN_FONT  /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
> 
> The font one above could be modified for [0-3] or similar, if we want to 
> catch tiny versus literally hidden fonts.
> 
> Cheers.
> 
> --- Amir
> 
There is T_HIDDEN_WORD on my sandbox 
(https://ruleqa.spamassassin.org/20190617-r1861495-n/T_HIDDEN_WORD/detail)
I have just committed a more generic version.
 Giovanni

Re: Rules for invisible div and 0pt font?

[Amir Caspi]

On Jun 17, 2019, at 2:17 PM, Amir Caspi  wrote:
> 
> rawbody   AC_MEDIA_DISPLAYNONE
> /@media[^{]*{[^}]*display\s*:\s*none\s*;/i
> 

Well, urgh, this particular rule wouldn't work well since it wouldn't capture 
classes within the @media block.  But something LIKE it.

--- Amir

Re: Rules for invisible div and 0pt font?

[Amir Caspi]

On Jun 17, 2019, at 1:45 PM, @lbutlr  wrote:
> 
> Would only be active if the width of the window is 900px or less. That can 
> include setting a display property to hidden or not.

One way of working around that, then, would be to ensure this is only within a 
div/span tag...

Maybe something like:

rawbody AC_HIDDEN_ELEMENT   /<(?:div|span|p)\s+[^>]+display\s*:\s*none\s*;/i

One could restrict this further by trying to capture the style= attribute but I 
don't think that's necessary.  This formulation should capture a display:none 
only in an inline style for div, span, and p elements.

HOWEVER, this formulation also leaves open the very easy workaround of defining 
a CSS style for hidden elements (e.g., putting display:none within a CSS class 
definition) and then setting the div/span/p element to that class.

So a DIFFERENT workaround would be to ensure that the display:none doesn't 
occur within a @media {} block... but because we can't use variable-length 
lookbehind, the only way I can think of doing that is to check for @media 
blocks WITH display:none, and the total number of display:none, and if the 
latter is larger, consider the rule to be hit.  I'm not sure how to compare the 
number of rule hits in SA... but if we wanted to do that, then:

rawbody AC_MEDIA_DISPLAYNONE/@media[^{]*{[^}]*display\s*:\s*none\s*;/i

Then create a meta that hits when AC_HIDDEN_ELEMENT > AC_MEDIA_DISPLAYNONE

Cheers.

--- Amir

Re: Rules for invisible div and 0pt font?

[@lbutlr]

On Jun 17, 2019, at 1:30 PM, Amir Caspi  wrote:
> Wouldn't that only be true for dynamic content that can actually evaluate the 
> screensize, and hence would require javascript?  Or is there a way of doing 
> this with static email content?  (I'm very well versed in HTML for web 
> browsers, but not as much for MUAs...)

Pretty sure in css you can display based on screen size (well, not screen per 
se, but display size) without resorting to javascript, but I am not positive.

@media (max-width: 900px) {
   … stuff
}

Would only be active if the width of the window is 900px or less. That can 
include setting a display property to hidden or not.

-- 
"Kill yourself and roll a rogue. We'll wait"

Re: Rules for invisible div and 0pt font?

[Amir Caspi]

On Jun 17, 2019, at 1:18 PM, Antony Stone 
 wrote:
> 
> If this feature *is* used for screenreaders, you could be creating a false 
> positive trap here...

You may well be right, hence the request to sandbox and see how it compares 
against masscheck.

On Jun 17, 2019, at 1:25 PM, @lbutlr  wrote:
> 
> Since display:none is a pretty common method for showing and hiding elements 
> depending on things like screen size, I would guess this is going to hit 
> mostly ham.

Wouldn't that only be true for dynamic content that can actually evaluate the 
screensize, and hence would require javascript?  Or is there a way of doing 
this with static email content?  (I'm very well versed in HTML for web 
browsers, but not as much for MUAs...)

The font-size, line-height, max-height, max-width of would almost certainly be 
pretty spammy, I would imagine.

Anyway, that's the whole point of sandboxing...

Cheers!

--- Amir

Re: Rules for invisible div and 0pt font?

[Amir Caspi]

On Jun 17, 2019, at 1:14 PM, Amir Caspi  wrote:
> 
> rawbody   AC_HIDDEN_FONT  /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
> 

Actually, based on another spample (https://pastebin.com/rrU2AsVT 
), let's modify this one -- the em/pt/px/% isn't 
required:

rawbody AC_HIDDEN_FONT  /font-size\s*:\s*0\s*(?:em|pt|px|%)?\s*;/

It might also be prudent to look for 0-height or 0-width line-height, 
max-height, max-width... so that would change the hidden-font to:

rawbody AC_HIDDEN_FONT  
/(?:font-size|line-height|max-height|max-width)\s*:\s*0\s*(?:em|pt|px|%)?\s*;/

And, looks like another rule might be useful:

rawbody AC_LARGE_NEG_INDENT /text-indent\s*:\s*-[0-9]{3,}(?:em|pt|px|%)\s*;/

This looks for a large negative text-indent, as is used in the spample linked 
above.

Cheers.

--- Amir

Re: Rules for invisible div and 0pt font?

[@lbutlr]

On Jun 17, 2019, at 1:14 PM, Amir Caspi  wrote:
> rawbody   AC_HIDDEN_ELEMENT   /display\s*:\s*none\s*;/

Since display:none is a pretty common method for showing and hiding elements 
depending on things like screen size, I would guess this is going to hit mostly 
ham.

-- 
It was easy to be a vegetarian by day. It was preventing yourself from
becoming a humanitarian at night that took the real effort.

Re: Rules for invisible div and 0pt font?

[Antony Stone]

On Monday 17 June 2019 at 21:14:36, Amir Caspi wrote:

> Hi all,
> 
> In reviewing today's FNs I came across the following spample:
> https://pastebin.com/9QQVwUY6
> 
> There is a div here with display:none, as well as font-size:0px.  The
> spample hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule
> relating to a hidden div or tiny font.
> 
> Does LOW_CONTRAST include font-size too small, or just color too light?  Is
> there a rule for matching display:none?

Is display:none ever used for instructiosn to screen readers for the blind / 
visually impaired?

I have no idea whether it is, but it's a potentially legitimate use which 
comes to mind.  If not, what is "display:none" actually for?

> If not, may I propose that the following rules be sandboxed?
> 
> rawbody   AC_HIDDEN_ELEMENT   /display\s*:\s*none\s*;/
> 
> rawbody   AC_HIDDEN_FONT  /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/
> 
> The font one above could be modified for [0-3] or similar, if we want to
> catch tiny versus literally hidden fonts.

If this feature *is* used for screenreaders, you could be creating a false 
positive trap here...


Antony.

-- 
Wanted: telepath.   You know where to apply.

   Please reply to the list;
 please *don't* CC me.

Rules for invisible div and 0pt font?

[Amir Caspi]

Hi all,

In reviewing today's FNs I came across the following spample:
https://pastebin.com/9QQVwUY6

There is a div here with display:none, as well as font-size:0px.  The spample 
hits HTML_FONT_LOW_CONTRAST but does not appear to hit any rule relating to a 
hidden div or tiny font.

Does LOW_CONTRAST include font-size too small, or just color too light?  Is 
there a rule for matching display:none?

If not, may I propose that the following rules be sandboxed?

rawbody AC_HIDDEN_ELEMENT   /display\s*:\s*none\s*;/

rawbody AC_HIDDEN_FONT  /font-size\s*:\s*0\s*(?:em|pt|px|%)\s*;/

The font one above could be modified for [0-3] or similar, if we want to catch 
tiny versus literally hidden fonts.

Cheers.

--- Amir