Re: Traffic filtering

[Marko Cupać]

On Mon, 30 Oct 2017 20:50:46 +
greg...@airmail.cc wrote:

> Hi,
> I'm new to this area, but I would like to filter some traffic.
> The goal is to keep people secure while web browsing, not to censure.
> And also enable better privacy, mainly stop "malware" and
> tracking/ads as restrictively as possible.
> 
> I have 3 questions, in case someone here has the time to answer me:
> 
> 1. What layers I should be filtering? Direct IP drop using pf,
> DNS drop with NSD/Unbound server, layer 7 with relayd, etc.

I'm filtering web traffic with squid, a http proxy. That way I can give
more information to users about reasons for restriction, not just
"request timeout" or "no dns record".

> 2. If the right approach is blacklisting domains, then what list
> do OpenBSD users recommend to use? People seem to be using these
> two, but I would like to know the opinion from OpenBSD users:
> http://www.malware-domains.com/files/
> https://hosts-file.net/?s=Download

I had good experience with http://www.shallalist.de/

> 3. There's any well designed tool that I can automatically update
> these lists (using pledge and signify, for example), or a simple shell
> script is enough?

ftp and reload service.

-- 
Before enlightenment - chop wood, draw water.
After  enlightenment - chop wood, draw water.

Marko Cupać
https://www.mimar.rs/

Re: Traffic Filtering

[10ejcpdjsc0]

OP here. I was reading more about it and you can
actually (mostly) block entire companies such as
ads networks, Google, Facebook, Akamai, Yahoo,
etc, using their AS number.
For example, use this tool to find the corporation:
https://www.ultratools.com/tools/asnInfo

Then get their IP list (substitute the "asn=x"
for the actual number):
https://www.enjen.net/asn-blocklist/index.php?asn=x=iplist=1

Use the IP list on pf(4) to drop everything.

You can also block entire countries using RIR:
http://lite.ip2location.com/database-ip-country
http://dev.maxmind.com/geoip/geoip2/geolite2/
http://www.ipdeny.com/ipblocks/data/countries/

Countries like China and Russia usually have a
bad log on attacks, tracking, ads, etc.

For Unbound, I've found these:
https://github.com/firehol/blocklist-ipsets/
https://github.com/StevenBlack/hosts/

--

P.S: Here's a list of ASN's you could want to block:

AS6432  - GOOGLE-FIBER - Google Fiber Inc., US
AS22577 - ADMOB-US - Google Inc., US
AS15169 - GOOGLE - Google Inc., US
AS36384 - GOOGLE-IT - Google Incorporated, US
AS36040 - YOUTUBE - Google Inc., US
AS36492 - GOOGLEWIFI - Google, Inc., US
AS41264 - GOOGLE-IT-RO-ISP, CH
AS45566 - GOOGLE-CORP-APAC-AS-AP AS number for Google Corporate Network 
in APAC, IN


AS32934 - FACEBOOK - Facebook, Inc., US
AS23455 - AKAMAI-AS - Akamai Technologies, Inc., US
AS21342 - AKAMAI-ASN2, EU
AS16702 - AKAMAI-AS - Akamai Technologies, Inc., US
AS22207 - AKAMAI-AS - Akamai Technologies, Inc., US
AS31377 - AKAMAI-BOS, US
AS23903 - AKAMAI-AS-BANGLORE Akamai Banglore Office ASN, IN
AS21399 - AKAMAI3, US
AS20189 - AKAMAI-AS - Akamai Technologies, Inc., US
AS33905 - AKAMAI-AMS, EU
AS43639 - AKAMAI-AMS2, NL
AS31109 - AKAMAI-LA, EU
AS31110 - AKAMAI-SJC, EU
AS34850 - AKAMAI-MUC, IR
AS1 - AKAMAI - Akamai Technologies, Inc., US
AS18680 - AKAMAI-AS - Akamai Technologies, Inc., US
AS35204 - AKAMAI-DUB, EU
AS39836 - AKAMAI-FRA, DE
AS35994 - AKAMAI-AS - Akamai Technologies, Inc., US
AS24319 - AKAMAI-TYO-AP Akamai Technologies Tokyo ASN, SG
AS23454 - AKAMAI-AS - Akamai Technologies, Inc., US
AS31108 - AKAMAI-VA, EU
AS34164 - AKAMAI-LON, GB
AS20940 - AKAMAI-ASN1, US
AS18717 - AKAMAI-AS - Akamai Technologies, Inc., US
AS35993 - AKAMAI-AS - Akamai Technologies, Inc., US

AS6182 - MICROSOFT-CORP-MSN-AS-4 - Microsoft Corporation, US
AS3598 - MICROSOFT-CORP-AS - Microsoft Corporation, US
AS8075 - MICROSOFT-CORP-MSN-AS-BLOCK - Microsoft Corporation, US
AS8072 - MICROSOFT-CORP-MSN-AS-BLOCK - Microsoft Corporation, US
AS8069 - MICROSOFT-CORP-MSN-AS-BLOCK - Microsoft Corporation, US
AS8068 - MICROSOFT-CORP-MSN-AS-BLOCK - Microsoft Corporation, US
AS13811 - MSLI - Microsoft Corporation, US
AS20046 - MICROSOFT-BOS - Microsoft Corporation, US
AS26222 - MS-DANGER - Microsoft Corporation, US
AS23468 - MICROSOFT-CORP-XBOX-ONLINE - Microsoft Corporation, US

AS7233 - YAHOO-US - Yahoo, US
AS5779 - YAHOO-DNB - Yahoo! Broadcast Services, Inc., US
AS7280 - YAHOO-FC - Yahoo! Inc., US
AS4694 - IDC Yahoo Japan Corporation, JP
AS2521 - IDC2521 Yahoo Japan Corporation, JP
AS2554 - IDC2554 Yahoo Japan Corporation, JP
AS7488 - IDC7488 Yahoo Japan Corporation, JP
AS55417 - YAHOO-SGA YAHOO! SGA, TW
AS40986 - YAHOO1-AS from AS28730 accept ANY, GB
AS22565 - YAHOO-NUQ - Yahoo, US
AS55517 - YAHOO-HKA YAHOO! HKA, HK
AS36752 - YAHOO-SP1 - Yahoo, US
AS36646 - YAHOO-NE1 - Yahoo, US
AS10310 - YAHOO-1 - Yahoo!, US
AS23816 - YAHOO Yahoo Japan Corporation, JP
AS34082 - YAHOO-AMA, GB
AS26101 - YAHOO-3 - Yahoo!, US
AS45915 - YAHOO-CORP-BWS-AS Yahoo! India Pvt Ltd., IN
AS45502 - YAHOO-CORP-MUMBAI-AP Yahoo Corp Network, KR
AS38689 - YHKR3-AS-KR KR3 Service Co,.Ltd., KR
AS24236 - YAHOO-BANGALORE-AS-AP Yahoo Bangalore Network Monitoring 
Center, HK

AS34010 - YAHOO-IRD, GB
AS36088 - YAHOO-BCST-AC2 - Yahoo, US
AS45863 - YAHOO-CORP-NDI-AS Yahoo! India Pvt Ltd., IN
AS17110 - YAHOO-US2 - Yahoo, US
AS10880 - YAHOO-AN2 - Yahoo, US
AS36129 - YAHOO-MAVEN - Yahoo, US
AS24376 - YAHOO-CN2-AP Yahoo China Datacenter, CN
AS14678 - YAHOO-HILLSBORO - Yahoo, US
AS10157 - YAHOO-AS-KR Yahoo! Korea, Corp., KR
AS55416 - YAHOO-KRA YAHOO! KRA, KR
AS24018 - YAHOO-BACKBONE-AP Yahoo Backbone Network, Asia Pacific, HK
AS36229 - YAHOO-YSM-SC8 - Yahoo! Inc., US
AS45501 - YAHOO-CORP-SG-AS-AP Yahoo Corp Network, SG
AS38072 - YAHOO-IN2-AS Yahoo! Web Services India Pvt Ltd., IN
AS55418 - YAHOO-ID1 YAHOO! ID1, SG
AS15635 - YAHOO-UKL, GB
AS43428 - YAHOO-ULS, GB
AS42173 - YAHOO-SWITZERLAND, CH
AS36647 - YAHOO-GQ1 - Yahoo, US
AS15896 - YAHOO-DEA, DE
AS24572 - YAHOO-JP-AS-AP Yahoo Japan, JP
AS24506 - YAHOO-TP2 YAHOO! TAIWAN, TW
AS26085 - YAHOO-2 - Yahoo!, US
AS14196 - YAHOO-CHA - Yahoo, US
AS23926 - YAHOO-JP3-AP JP DC, JP

Re: Traffic filtering

[Erik van Westen]

Op 30-10-2017 om 22:37 schreef x9p:
>
>> I use the blocklists from emergingthreats.net. Is already in a format
>> that  Works wonderfully.
>>
>> http://rules.emergingthreats.net/fwrules/emerging-PF-ALL.rules
>
> Good to use HTTPS to avoid someone tampering with the list via DNS/etc..

So use https://rules.emergingthreats.net/fwrules/emerging-PF-ALL.rules
instead... I won't stop you. :) What are the chances that someone will
be able to realisticly tamper with the traffic? Very close to zero given
the setup. The chances that the other side is tampered with are much,
much higher. You might as well not rely on external sources.
Of course you are running your own DNS resolver and not relying on your
provider, are you?

>
>> Just fetch them through a cron job, include them in pf.conf and reload
>> pf.conf. And yes, you would have to trust...
>
> Is a nice idea to whitelist the IP address/range where you connect
> from, if loading external rules made by somebody else, so you do not
> get locked out of your own box (happened once on a friday, not funny).

Won't happen, thanks for the warning though. I connect from the inside
(always access) to the outside, and when connecting from the outside it
will be over IPv6. The list is IPv4.

Erik

Re: Traffic filtering

[x9p]

I use the blocklists from emergingthreats.net. Is already in a format
that  Works wonderfully.

http://rules.emergingthreats.net/fwrules/emerging-PF-ALL.rules


Good to use HTTPS to avoid someone tampering with the list via DNS/etc..


Just fetch them through a cron job, include them in pf.conf and reload
pf.conf. And yes, you would have to trust...


Is a nice idea to whitelist the IP address/range where you connect from, 
if loading external rules made by somebody else, so you do not get 
locked out of your own box (happened once on a friday, not funny).


cheers.

x9p

Re: Traffic filtering

[Sterling Archer]

I use these lists myself:

http://sysctl.org/cameleon/hosts
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
https://hosts-file.net/ad_servers.txt
https://mirror1.malwaredomains.com/files/justdomains
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
http://someonewhocares.org/hosts

I run them through a shell script that creates an unbound config file
that redirects the requests to a dedicated httpd that returns an
HTTP 204 for anything except images. Those get a 1x1 gif back.

The only issue I have is with sites that redirect links to a tracker,
but I can live with taht.


On Mon, Oct 30, 2017 at 9:50 PM,   wrote:
> Hi,
> I'm new to this area, but I would like to filter some traffic.
> The goal is to keep people secure while web browsing, not to censure.
> And also enable better privacy, mainly stop "malware" and
> tracking/ads as restrictively as possible.
>
> I have 3 questions, in case someone here has the time to answer me:
>
> 1. What layers I should be filtering? Direct IP drop using pf,
> DNS drop with NSD/Unbound server, layer 7 with relayd, etc.
>
> 2. If the right approach is blacklisting domains, then what list
> do OpenBSD users recommend to use? People seem to be using these
> two, but I would like to know the opinion from OpenBSD users:
> http://www.malware-domains.com/files/
> https://hosts-file.net/?s=Download
>
> 3. There's any well designed tool that I can automatically update
> these lists (using pledge and signify, for example), or a simple shell
> script is enough?
>
>
> Any advice is welcome.
>



-- 
:wq!

Re: Traffic filtering

[Erik van Westen]

Op 30-10-2017 om 21:50 schreef greg...@airmail.cc:
> Hi,
> I'm new to this area, but I would like to filter some traffic.
> The goal is to keep people secure while web browsing, not to censure.
> And also enable better privacy, mainly stop "malware" and
> tracking/ads as restrictively as possible.
>
> I have 3 questions, in case someone here has the time to answer me:
>
[snip]
> 2. If the right approach is blacklisting domains, then what list
> do OpenBSD users recommend to use? People seem to be using these
> two, but I would like to know the opinion from OpenBSD users:
> http://www.malware-domains.com/files/
> https://hosts-file.net/?s=Download
> 3. There's any well designed tool that I can automatically update
> these lists (using pledge and signify, for example), or a simple shell
> script is enough?

I use the blocklists from emergingthreats.net. Is already in a format
that  Works wonderfully.

http://rules.emergingthreats.net/fwrules/emerging-PF-ALL.rules

Just fetch them through a cron job, include them in pf.conf and reload
pf.conf. And yes, you would have to trust...

Good luck.

Erik

Traffic filtering

[gregfod]

Hi,
I'm new to this area, but I would like to filter some traffic.
The goal is to keep people secure while web browsing, not to censure.
And also enable better privacy, mainly stop "malware" and
tracking/ads as restrictively as possible.

I have 3 questions, in case someone here has the time to answer me:

1. What layers I should be filtering? Direct IP drop using pf,
DNS drop with NSD/Unbound server, layer 7 with relayd, etc.

2. If the right approach is blacklisting domains, then what list
do OpenBSD users recommend to use? People seem to be using these
two, but I would like to know the opinion from OpenBSD users:
http://www.malware-domains.com/files/
https://hosts-file.net/?s=Download

3. There's any well designed tool that I can automatically update
these lists (using pledge and signify, for example), or a simple shell
script is enough?


Any advice is welcome.

tip for inter-KVM VMs traffic filtering with PF running on separate box

[Jiri B]

Hi,

I'm curious how to filter inter-VMs (running on Linux KVM host) traffic
on a remote bare-metal host running OpenBSD and PF. Any tip?

So, there would be a Linux KVM host running various VMs and separate
OpenBSD box and I'd like to achieve that all traffic betweens those VMs
running on that Linux box is sent to OpenBSD box which does PF and "switching".

libvirt docs says (about vepa-type bridging on Linux):

~~~
vepa
All VMs' packets are sent to the external bridge. Packets
whose destination is a VM on the same host as where the packet
originates from are sent back to the host by the VEPA capable bridge
(today's bridges are typically not VEPA capable).
~~~

Problem is, as they say, many bridges/network switches are not VEPA capable.

So what could I do?

Could I use vxlan/openvswitch and connect it to OpenBSD...

I'm little bit lost about all pieces in this area.

Thanks for you tips and comments.

j.